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I n writing this book, I was guided by my long-standing experience and interest in 
teaching discrete mathematics. For the student, my purpose was to present ma¬ 
terial in a precise, readable manner, with the concepts and techniques of discrete 
mathematics clearly presented and demonstrated. My goal was to show the relevance 
and practicality of discrete mathematics to students, who are often skeptical. I wanted to 
give students studying computer science all the mathematical foundations they need for 
their future studies; I wanted to give mathematics students an understanding of impor¬ 
tant mathematical concepts together with a sense of why these concepts are important 
for applications. And I wanted to accomplish these goals without watering down the 
material. 

For the instructor, my purpose was to design a flexible, comprehensive teaching tool 
using proven pedagogical techniques in mathematics. I wanted to provide instructors 
with a package of materials that they could use to teach discrete mathematics effectively 
and efficiently in the most appropriate manner for their particular set of students. I hope 
that I have achieved these goals. 

I have been extremely gratified by the tremendous success of this text. The many im¬ 
provements in the fourth edition have been made possible by the feedback and sugges¬ 
tions of a large number of instructors and students at many of the more than 400 schools 
where (his book has been successfully used. There are many enhancements in this edi¬ 
tion. The ancillary package has been enriched, and a companion Web site provides help¬ 
ful material, making it easier for students and instructors to achieve their goals. 

This text is designed for a one- or two-term introductory discrete mathematics 
course to be taken by students in a wide variety of majors, including mathematics, 
computer science, and engineering. College algebra is the only explicit prerequisite. 


Goals of a Discrete Mathematics Course 


A discrete mathematics course has more than one purpose. Students should learn a par¬ 
ticular set of mathematical facts and how to apply them; more importantly, such a course 
should teach students how to think mathematically. To achieve these goals, this text 
stresses mathematical reasoning and the different ways problems are solved. Five impor¬ 
tant themes are interwoven in this text: mathematical reasoning, combinatorial analysis, 
discrete structures, algorithmic thinking, and applications and modeling. A successful 
discrete mathematics course should carefully blend and balance all five themes, 

1. Mathematical Reasoning: Students must understand mathematical reasoning in 
order to read, comprehend, and construct mathematical arguments. This text starts 
with a discussion of mathematical logic, which serves as the foundation for the 
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subsequent discussions of methods of proof. The technique of mathematical in¬ 
duction is stressed through many different types of examples of such proofs and a 
careful explanation of why mathematical induction is a valid proof technique. 

2. Combinatorial Amlysis: An important problem-solving skill is the ability to count 
or enumerate objects. The discussion of enumeration in this book begins with the 
basic techniques of counting. The stress is on performing combinatorial analysis 
to solve counting problems, not on applying formulae, 

3. Discrete Structures: A course in discrete mathematics should teach students how 
to work with discrete structures, which are the abstract mathematical structures 
used to represent discrete objects and relationships between these objects. These 
discrete structures include sets, permutations, relations, graphs, trees, and finite- 
state machines. 

4. Algorithmic Thinking: Certain classes of problems are solved by the specification 
of an algorithm. After an algorithm has been described, a computer program can 
be constructed implementing it. The mathematical portions of this activity, which 
include the specification of the algorithm, the verification that it works properly, 
and the analysis of the computer memory and time required to perform it, are all 
covered in this text. Algorithms are described using both English and an easily 
understood form of pseudocode. 

5. Applications and A3 adding: Discrete mathematics has applications to almost ev¬ 
ery conceivable area of study. There are many applications to computer science 
and data networking in this text, as well as applications to such diverse areas 
as chemistry, botany, zoology, linguistics, geography, business, and the Internet. 
These applications are natural and important uses of discrete mathematics and 
arc not contrived. Modeling with discrete mathematics is an extremely important 
problem-solving skill, which students have the opportunity to develop by con¬ 
structing their own models in some of the exercises in the book. 


Why a Fourth Edition? 


The third edition of this book has been used successfully at over 400 schools in the 
United States, at dozens of Canadian universities, and at universities in Europe, Asia, 
and Oceania, Many students and professors like the third edition as it is. Why then, do 
we need a fourth edition? This is a valid question deserving a careful answer. 

First, although the third edition has been extremely effective, many instructors have 
asked for specific improvements. Many have wanted changes to the text, additional or 
clarified examples, snore exercises of a certain type, or new topics covered. In this new 
edition I have improved the book by taking into account the numerous suggestions I 
have received. The changes I have made at the request of users make this a better text. 

Second, discrete mathematics is an active subject. There are many new discoveries 
made every year, and some of these can be reflected in a text. So, I have included dis¬ 
coveries made after the publication of the third edition. (Subsequent discoveries will be 
included in later printings of this edition whenever possible and noted on the companion 
Web site.) 

Third, since the publication of the third edition, the Internet has become extraordi¬ 
narily important and useful. In this edition you will find examples and exercises relating 
applications of discrete mathematics to the structure of the Internet itself. And with this 
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edition there is an extensive Web site that supplements the text in meaningful ways, 
offering additional material for students and instructors and providing a gateway for 
learning more about discrete mathematics by providing links to relevant sites on the 
Web. However, since many people will choose not to use the Web in conjunction with 
wb this course, the text includes icons indicating the inclusion of Web links in the annotated 
Web Guide on the Web site for this book. 

The following list highlights some of the changes in this edition that make the book 
more effective. 

NEW TOPIC COVERAGE 

■ Big-Omega and big-Theta notation are now covered, in addition to big-0 
notation. 

■ New topics in probability theory include the variance of a random variable and 
Chebyshev’s inequality. Also, the Monty Hall three-door problem is now dis¬ 
cussed in the text. 

■ The halting problem is now treated, including a proof that it is unsolvable. 

■ The traveling salesman problem is discussed. 

EXPANDED TOPIC COVERAGE 

■ Additional material on mathematical logic and mathematical reasoning has been 
added. New examples show how to translate between quantified statements and 
English. The discussion of rules of inference has been enhanced. In particular, 
rules of inference for quantified statements are now explicitly covered, and ex¬ 
amples illustrating how rules of inference are used have been added. 

a Coverage of the floor and ceiling functions has been enhanced. 

■ Generating functions are now treated in a separate section in the main body of 
the text, expanding the coverage previously found in the appendix. The focus of 
this section is to show how generating functions can be used to solve counting 
problems, solve recurrence relations, and prove combinatorial identities. 

■ Nonhomogeneous linear recurrence relations with constant coefficients are now 
discussed in the text, rather than in an exercise set. 

■ The topic of integer sequences has extended coverage; examples and exercises 
involving identifying possible formulas for the terms of a sequence from its initial 
terms have been added. 

■ New biographies have been added, including those for Peirce, Chebyshev, 
Knuth, Hardy, Ramanujan, Tukey, Sloane, and Mersenne, 

UP-TO-DATE, MODERN EXAMPLES 

■ Examples have been added at some key points in the text to help explain im¬ 
portant concepts that have proved troublesome to students and to make the book 
more interesting. 

■ Examples and exercises illustrating the application of discrete mathematics to 
the protocols and network architecture of the Internet have been added. These 
additions include counting problems involving Internet addresses and Internet 
Protocol packets; the topic of Boolean searching, used by Internet search engines; 
and an example about how spanning trees are used in IP multicasting have been 
added. 
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■ Material has been added to the text which demonstrates that discrete mathematics 
is an active subject with many open questions and with new discoveries. For 
example, Mersenste primes are now covered, including the discoveries of new 
primes in 1997 and 1998; the range for which the Goldbaeb conjecture has been 
verified is discussed; and the variation of the Tower of Hanoi puzzle with four 
pegs is described, 

EXPANDED EXERCISE SETS 

■ More than 500 new exercises have been added, including both routine and chal¬ 
lenging ones, as requested by instructors who used the third edition, as well as 
exercises based on logical and mathematical puzzles. New blocks of exercises 
develop key concepts in a series of steps. New exercises ensure that there are 
both odd- and even-numbered exercises of important exercise types. There are 
also more exercises that depend on the previous study of calculus; these are ex¬ 
plicitly noted as usual and can be easily avoided if so desired. 

WEB Sl'PPORT 

■ A Web site has been developed to supplement the text for both students and 
instructors. This Web site contains a wide range of features (see page xix), in¬ 
cluding an annotated Web Guide to relevant sites on the Internet, that is keyed 
to the text. This guide will be kept current and updated regularly during the life 
of this edition. 

■ An icon has been placed at points in the text whenever the Web Guide includes 
annotated links to Web sites pertinent to the material under discussion, (More 
than 200 different links are in the guide.) These sites include additional in¬ 
formation about concepts and applications, biographies, the latest discoveries, 
downloadable source code, interactive applets, animated algorithms, and other 
interesting material. 


Special Features 

ACCESSIBILITY This text has proven to be easily read and understood by be¬ 
ginning students. There are no mathematical prerequisites beyond college algebra for 
almost all of this text. The few places in the book where calculus is referred to are ex¬ 
plicitly noted. Most students should easily understand the pseudocode used in the text 
to express algorithms, regardless of whether they have formally studied programming 
languages. There is no formal computer science prerequisite, 

Each chapter begins at an easily understood and accessible level. Once basic math¬ 
ematical concepts have been carefully developed, more difficult material and applica¬ 
tions to other areas of study are presented. 

FLEXIBILITY This text has been carefully designed for flexible use. The depen¬ 
dence of chapters on previous material has been minimized. Each chapter is divided 
into sections of approximately the same length, and each section is divided into subsec¬ 
tions that form natural blocks of material for teaching. Instructors can easily pace their 
lectures using these blocks. 


PDF created with pdfFactory trial version www.pdffactorv.com 





Preface xiii 

WRITING STYLE The writing style in this book is direct and pragmatic. Precise 
mathematical language is used without excessive formalism and abstraction. Notation 
is introduced and used when appropriate. Care has been taken to balance the mix of 
notation and words in mathematical statements. 

EXTENSIVE CLASSROOM USE This book has been used at over 400 schools, 
and more than 325 have used it more than once. The feedback from instructors and stu¬ 
dents at many of the schools has helped make the fourth edition an even more successful 
teaching tool than previous editions. 

MATHEMATICAL RIGOR AND PRECISION All definitions and theorems in 
this text are stated extremely carefully so that students will appreciate the precision of 
language and rigor needed in mathematics. Proofs are motivated and developed slowly; 
their steps are all carefully justified. Recursive definitions are explained and used 
extensively. 

FIGURES AND TABLES This text contains more than 550 figures. The figures are 
designed to illustrate key concepts and steps of proofs. Color has been carefully used 
in figures to illustrate important points. Whenever possible, tables have been used to 
summarize key points and illuminate quantitative relationships. 

WORKED EXAMPLES Over 650 examples are used to illustrate concepts, relate 
different topics, and introduce applications. In the examples, a question is first posed, 
then its solution is presented with the appropriate amount of detail. 

APPLICATIONS The applications included in this text demonstrate the utility of 
discrete mathematics in the solution of real-world problems. This text includes appli¬ 
cations to a wide variety of areas, including computer science, data networking, psy¬ 
chology, chemistry, engineering, linguistics, biology, business, and the Internet. 

ALGORITHMS Results in discrete mathematics are often expressed in terms of 
algorithms; hence, key algorithms are introduced in each chapter of the book. These 
algorithms are expressed in words and in an easily understood form of structured pseu¬ 
docode, which is described and specified in Appendix A.2, The computational com¬ 
plexity of the algorithms in the text is also analyzed at an elementary' level, 

HISTORICAL INFORMATION The background of many topics is succinctly de¬ 
scribed in the text. Brief biographies of more than 55 mathematicians and computer 
scientists are included as footnotes. These biographies include information about the 
lives, careers, and accomplishments of these important contributors to discrete math¬ 
ematics. In addition, numerous historical footnotes are included that supplement the 
historical information in the main body of the text. 

KEY TERMS AND RESULTS A list of key terms and results follows each chapter. 
The key terms include only the most important that students should learn, not every' 
term defined in the chapter. 

EXERCISES There are over 3(300 exercises in the text. There are many different 
types of questions posed. There is an ample supply of straightforward exercises that 
develop basic skills, a large number of intermediate exercises, and many challenging 
exercises, Exercises are stated clearly and unambiguously, and all are carefully graded 
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for level of difficulty. Exercise sets contain special discussions, with exercises, that 
develop new concepts not covered in the text, permitting students to discover new ideas 
through their own work. 

Exercises that are somewhat more difficult than average are marked with a single 
star; those that are much more challenging are marked with two stars. Exercises whose 
solutions require calculus are explicitly noted. Exercises that develop results used in the 
text are clearly identified with the symbol «s\ Answers or outlined solutions to all odd- 
numbered exercises are provided at the back of the text. The solutions include proofs 
in which most of the steps are clearly spelled out. 

REVIEW QUESTIONS A set of review questions is provided at the end of each 
chapter. These questions are designed to help students focus their study on the most 
important concepts and techniques of that chapter. To answer these questions students 
need to write long answers, rather than just perform calculations or give short replies. 

SUPPLEMENTARY EXERCISE SETS Each chapter is followed by a rich and 
varied set of supplementary exercises. These exercises are generally more difficult than 
those in the exercise sets following the sections. The supplementary exercises reinforce 
the concepts of the chapter and integrate different topics more effectively. 

COMPUTER PROJECTS Each chapter is followed by a set of computer projects. 
The approximately 150 computer projects tie together what students may have learned 
in computing and in discrete mathematics. Computer projects that are more difficult 
than average, from both a mathematical and a programming point of view, are marked 
with a star, and those that are extremely challenging are marked with two stars. 

COMPUTATIONS AND EXPLORATIONS A set of computations and explo¬ 
rations is included at the conclusion of each chapter. These exercises (approximately 
100 in total) are designed to be completed using existing software tools, such as pro¬ 
grams that students or instructors have written or mathematical computation packages 
such as MAPLE or Mathematiea, Many of these exercises give students the opportu¬ 
nity to uncover new facts and ideas through computation. (Some of these exercises are 
discussed in the companion volume, Exploring Discrete Mathematics with MAPLE.) 

WRITING PROJECTS Each chapter is followed by a set of writing projects. To 
do these projects students need to consult the mathematical literature. Some of these 
projects are historical in nature and may involve looking up original sources. Others 
are designed to serve as gateways to new topics and ideas. All are designed to expose 
students to ideas not covered in depth in the text. These projects tie together mathe¬ 
matical concepts and the writing process and help expose students to possible areas 
for future study. (Suggested references for these projects can be found in the Student 
Solutions Guide.) 

APPENDIXES There are two appendixes to the text. The first covers exponential 
and logarithmic functions, reviewing some basic material used heavily in the course; 
the second specifies the pseudocode used to describe algorithms in this text. 

SUGGESTED READING A list of suggested readings for each chapter is provided 
in a section at the end of the text. These suggested readings include books at or below 
the level of this text, more difficult books, expository articles, and articles in which 
discoveries in discrete mathematics were originally published. 
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How To Use This Book 

This test has been carefully written and constructed to support discrete mathematics 
courses at several levels and with differing foci. The following table identifies the core 
and optional sections. An introductory one-term course in discrete mathematics at the 
sophomore level can be based on the core sections of the text, with other sections cov¬ 
ered at the discretion of the instructor, A two-term introductory course could include 
all the optional mathematics sections in addition to the core sections. A course with a 
strong computer science emphasis can be taught by covering some or all of the optional 
computer science sections. 


Chapter 

Core Sections 

Optional Computer 
Science Sections 

Optional Mathematics 
Sections 

1 

1.1-1.8 (as needed) 



2 

2.1-2.3, 2.6 (as needed) 

2.4 

2.5 

3 

3.1-3.3 

3.4, 3.5 


4 

4,1- 4.4 

4.7 

4.5,4.6 

5 

5.1, 5.5 

5.3 

5.2,5.4,5.6 

6 

6.1,6.3, 6.5 

6.2 

6.4,6.6 

7 

7.1-7.5 


7.6-7.8 

8 

8.1 

8.2-8,4 

8.5, 8.6 

9 


9.1-9.4 


10 


10.1-10.5 



Instructors using this book can adjust the level of difficulty of their course by either 
choosing to cover or to omit the more challenging examples at the end of sections, as 
well as die more challenging exercises. The dependence of chapters on earlier chapters 
is shown in the following chart. 


Chapter i 

I 

Chapter 2 

I 

Chapter 3 


Chapter 4 



Chapter 5 Chapter 6 Chapter 7 Chapter 9 Chapter 10 

I 

Chapter 8 
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Ancillaries 


STUDENT SOLUTIONS GUIDE This student manual, available separately, con¬ 
tains Jw// solutions to all the odd-numbered problems in the exercise sets. These solu¬ 
tions explain why a particular method is used and why it works. For some exercises, one 
or tw o other possible approaches are described to show' that a problem can be solved in 
several different ways. Suggested references for the writing projects found at the end 
of each chapter are also included in this volume. The guide contains a guide to writing 
proofs and a list of common mistakes students make in discrete mathematics. It also 
includes sample tests and a sample crib sheet for each chapter, both designed to help 
students prepare for exams. Students find this guide extremely useful. 

INSTRUCTOR’S RESOURCE GUIDE This manual contains full solutions to 
even-numbered exercises in the text. It also provides suggestions on how to teach the 
material in each chapter of the book, including the points to stress in each section and 
how to put the material into perspective. Furthermore, the manual contains a test bank 
of sample examination questions for each chapter, including some sample tests as well 
as the solutions to the sample questions. Finally, sample syllabi are presented. 

APPLICATIONS OF DISCRETE MATHEMATICS This ancillary is a separate 
text that can be used either in conjunction with the text or independently. It contains 
more than 20 chapters (each with its own set of exercises) written by instructors who 
have used the text. Following a common format similar to that of the text, the chapters 
in this book can be used as a text for a separate course, for a student seminar, or for 
a student doing independent study. Subsequent editions of this ancillary are planned 
that will broaden the range of applications covered. Instructors are invited to submit 
additional applications for possible inclusion in later versions. 

TEST BANK An extensive test bank of more than 1300 questions is available for use 
on either Windows or Macintosh systems. Instructors can use this software to create their 
own tests by selecting questions of their choice or by random selection. Instructors can 
add their own headings and instructions, print scrambled versions of the same test, and 
edit the existing questions or add their own. A printed version of this test bank, including 
the questions and their answers, is included in the Instructor’s Resource Guide. 

EXPLORING DISCRETE MATHEMATICS AND ITS APPLICATIONS WITH 
MAPLE This ancillary is a separate book designed to help students use the MAPLE 
computer algebra system to do a wide range of computations in discrete mathematics. 
For each chapter of this text, this new ancillary includes the following: a description 
of relevant MAPLE functions and how they are used, MAPLE programs that carry out 
relevant computations, suggestions and examples showing how MAPLE can be used 
for the computations and explorations at the end of each chapter, and exercises that can 
be worked using MAPLE. 
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THE COMPANION 
WEB SITE 



A n extensive companion Web site has been developed and will be main¬ 
tained and improved on a continuing basis. The URL for this site is 
http://www.nfohe.corn/rosen. Following this URL takes you to a page that 
provides access to five different sections of the Web site: 

■ About the Book 
■ Instructor Resources 
■ Student Resources 
■ Web Guide for Discrete Mathematics 
■ Supplementary Resources 

Bach section will be in place with the publication of this new edition, although addi¬ 
tional material will be added later, 

The About the Book section includes basic information about the textbook and its 
anciliaries. It also contains an errata list and an e-mail address for the submission of 
errata and suggestions. 

The Instructor Resources section is a secure portion of the Web site. It contains 
valuable tools and resources to supplement both the text and the discrete mathematics 
teaching experience. 

The Student Resources section contains helpful reference and supplemental mate¬ 
rial to enhance students’ learning experience. 

The Web Guide for Discrete Mathematics section includes annotated links to rel¬ 
evant Web sites anchored to the Web icons in the text. ( Links are included wherever 
the icon is found.) The links in this guide can be used to access sites that provide bi¬ 
ographies, additional material on topics covered in the text, information on the latest 
discoveries, animated algorithms, downloadable source code, and so on. 

The Supplementary Resources section, intended for use by both students and in¬ 
structors, includes supplementary educational material, organized by chapter. This ma¬ 
terial is designed to clarify and expand on material in the text. 


xix 
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TO THE STUDENT 



W hat is discrete mathematics? Discrete mathematics is the part of mathemat¬ 
ics devoted to the study of discrete objects. (Here discrete means consisting 
of distinct or unconnected elements.) The kind of problems solved using 
discrete mathematics include; 

■ How many ways are there to choose a valid password on a computer system? 

■ What is the probability of winning a lottery? 

■ Is there a link between two computers in a network? 

■ What is the shortest path between two cities using a transportation system? 

■ How can a list of integers be sorted so that the integers are in increasing order? 

■ How many steps are required to do such a sorting? 

■ How can a circuit that adds two integers be designed? 

■ How many valid Internet addresses are there? 

You will learn the discrete structures and techniques needed to solve problems such as 
these. 

More generally, discrete mathematics is used whenever objects are counted, when 
relationships between finite (or countable) sets are studied, and when processes involv¬ 
ing a finite number of steps are analyzed. A key reason for the growth in the importance 
of discrete mathematics is that information is stored and manipulated by computing ma¬ 
chines in a discrete fashion. 

There are several important reasons for studying discrete mathematics. First, 
through this course you can develop your mathematical maturity, that is, your ability 
to understand and create mathematical arguments. You will not get very' far in your 
studies in the mathematical sciences without these skills, 

Second, discrete mathematics is the gateway to more advanced courses in all 
parts of the mathematical sciences. Discrete mathematics provides the mathematical 
foundations for many computer science courses, including data structures, algorithms, 
database theory', automata theory, formal languages, compiler theory, computer secu¬ 
rity, and operating systems. Students find these courses much more difficult when they 
have not had the appropriate mathematical foundations from discrete math. One student 
has sent me an electronic mail message to tell me that she used the contents of this 
book in every computer science course she took! 

Math courses based on the material studied in discrete mathematics include logic, 
set theory', number theory, linear algebra, abstract algebra, combinatorics, graph theory', 
and probability theory (the discrete part of the subject). 

Also, discrete mathematics contains the necessary mathematical background 
for solving problems in operations research (including many discrete optimization 
techniques), chemistry, engineering, biology, and so on. In the text, we will study 
applications to some of these areas. 

xx i 
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XXII 


TO THE STUDENT 


I would like to offer some helpful advice to students about how best to Feam discrete 
mathematics. You will leam the most by working exercises, I suggest, you do as many as 
you possibly can, including both the exercises at the end of each section of the text and 
the supplementary exercises at the end of each chapter. Always try exercises yourself 
before consulting the answers at the end of the book or in the Student Solutions Guide, 
Only after you have put together a solution, or you find yourself at an impasse, should 
you look up die suggested solution. At that point you will find the discussions in the 
Student Solutions Guide most helpful. When doing exercises, remember that the more 
difficult ones are marked as described in the following table. 


Key to the Exercises 

No marking 

A routine exercise 

* 

A difficult exercise 

** 

An extremely challenging exercise 

ri- 

An exercise containing a result used 
in the book 

(Calculus required) 

An exercise whose solution requires 
the use of limits or concepts from 
differential or integral calculus 


Finally, I encourage you to explore discrete mathematics beyond what you see in 
the book. An excellent starting place is the Web Guide for Discrete Mathematics that 
can be found on the Web site for this book. The URL is http://www.mhhe.com/rosen. 

Kenneth H. Rosen 
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The Foundations: Logic, 
Sets, and Functions 



his chapter reviews the foundations of discrete mathematics. Three important 



topics are covered: logic, sets, and functions. The rules of logic specify the 


A precise meaning of mathematical statements, For instance, the rules give us the 
meaning of such statements as, “There exists an integer that is greater than 100 that is 
a power of 2,” and, “For every integer n the sum of the positive integers not exceeding 
n is n{n 4 13/2." Logic is the basis of all mathematical reasoning, and it has practical 
applications to the design of computing machines, to artificial intelligence, to computer 
programming, to programming languages, and to other areas of computer science 

Much of discrete mathematics is devoted to t ie study of discrete structures, which 
are used to represent discrete objects. All discrete structures are built up from sets, 
which arc collections of objects. Examples of discrete structures built up from sets 
include combinations, which are unordered collections of objects used extensively in 
counting; relations, which are sets of ordered pairs that represent relationships between 
objects; graphs, which are sets of vertices and edges that connect vertices; and finite 
state machines, which axe used to model computing machines. 

The concept of a function is extremely important in discrete mathematics. A func¬ 
tion assigns to each element of a set precisely one element of a set. Such useful struc¬ 
tures as sequences and strings are special types of functions. Functions axe used to 
represent the number of steps a procedure uses to solve a problem. The analysis ot al¬ 
gorithms uses terminology and concepts related to the growth of functions. Recursive 
functions, defined by specifying their values at positive integers in terms of their values 
at smaller positive integers, are used to solve many counting problems. 


INTRODUCTION 


The rules of logic give precise meaning to mathematical statements. These rules are 
used to distinguish between valid and invalid mathematical arguments. Since a major 
goat of this book is to teach the reader how 7 to understand and how to construct correct 
mathematical arguments, we begin our study of discrete mathematics with an introduc¬ 
tion to logic. 

In addition to its importance in understanding mathematical reasoning, logic has 
numerous applications in computer science. These rules are used in the design of com¬ 
puter circuits, the construction of computer programs, the verification of the correctness 
of programs, and in many other ways. We will discuss each of these applications in the 
following chapters. 
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2 1 / The Foundations: Logic, Sets, and Functions 


PROPOSITIONS 

Our discussion begins with an introduction to the basic building blocks of logic- 
propositions. A proposition is a statement that is either true or false, but not both. 


EXAMPLE 1 All the following statements are propositions. 

1. Washington, D.C., is the capital of the United States of America. 

2. Toronto is the capital of Canada. 

3. 1-1 =2. 

4. 2 f 2 - 3. 

Propositions 1 and 3 are true, whereas 2 and 4 arc false, ■ 

Some sentences that are not propositions are given in the next example, 

EXAMPLE 2 Consider the following sentences. 

1. What time is it? 

2. Read this carefully. 

3. jt + J = 2. 

4. X + y = z. 

Sentences 1 and 2 are not propositions because they are not statements. Sentences 3 
and 4 are not propositions because they are neither true nor false, since the variables in 
these sentences have not been assigned values. Various ways to form propositions from 
sentences of this type will be discussed in Section 1.3. ■ 


letters are used to denote propositions, just as letters are used to denote variables. 

The conventional letters used for this purpose are p, q, r, s .The truth value of a 

proposition is true, denoted by T, if it is a true proposition and false, denoted by F, if it 
is a false proposition. 

We now turn our attention to methods for producing new propositions from those 
web 1,1 ;il we a,read > have. These methods were discussed by the English mathematician 
George Boole in ! 854 in his book The Laws of Thought. Many mathematical statements 
are constructed by combining one or more propositions. New propositions, called com¬ 
pound propositions, are funned from existing propositions using logical operators. 


DEFINITION 1 . Let p be a proposition. The statement 
“It is not the case that p." 

is another proposition, called the negation of p. The negation of p is denoted by -i p, 
The proposition -i p is read “not p " 
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1.1 Logic 3 


EXAMPLE 3 


web 


Find the negation of the proposition 
“Today is Friday” 
and express this in simple English. 

Solution: The negation is 

“It is not the case that today is Friday.” 

This negation can be more simply expressed by 

“Today is not Friday.” ■ 


Remark: Strictly speaking, sentences involving variable times such as those in Exam¬ 
ple 3 are not propositions unless a fixed time is assumed. The same holds for variable 
places unless a fixed place is assumed and for pronouns unless a particular person is 
assumed. 

A truth table displays the relationships between the truth values of propositions. 
Truth tables are especially valuable in the determination of the truth values of proposi¬ 
tions constructed from simpler propositions. Table 1 displays all possible truth values 
of a proposition and the corresponding truth values of its negation. 

The negation of a proposition can also be considered die result of the operation of 
the negation operator on a proposition. The negation operator constructs a new propo¬ 
sition from a single existing proposition. We will now introduce the logical operators 
that are used to form new propositions from two or more existing propositions. These 
logical operators are also called connectives. 


TAHI.K I The Truth Table 
for the Negation of a Proposition. 

P 

-'P 

T 

F 

F 

T 


George Book! 1815-1864). George Bode, the son of a cobbler, was bom in Lincoln. England, in Novem¬ 
ber 1815. Because of bis family’s difficult financial situation, Boole had to struggle to educate himself while 
supporting his family. Nevertheless, he became one of she most important mathematicians of die 1800s Al¬ 
though he considered a career as a clergyman, he decided instead to go into teaching and soon afterward 
opened a school of his own. In his preparation for teaching mathematics, Boole—unsatisfied with textbooks 
of his day—decided to read the works of the great mathematicians. While reading papers of the great French 
mathematician Lagrange, Boole made discoveries in the calculus of variations, the branch of analysis deal¬ 
ing with finding curves and surfaces optimizing certain parameters. 

In 1848 Book published The Mathematical Analysis ofLogic, the first of his contributions to symbolic 
logic in 1849 he was appointed professor of mathematics at Queen’s College in Cork, Ireland. In 1854 he 
published The Laws aj Thought, his most famous work, In this book Boole introduced what is now called 
Boolean algebra in his honor. Boole wrote textbooks on differential equations and on difference equations 
that were used in Great Britain unlil the end of the nineteenth century. Boole married in 1855; his wife 
was the niece of the professor of Greek at Queen's College Tn 1 Rri4 Boole died from pneumonia, which he 
contracted as a result of keeping a lecture engagement even though he was snaking wd from a rainstorm. 
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I i The Founilalkim: ! .ogk. Sets, and Functions 


EXAMPLE 4 


DEFINITION 2. Let p aiKl^be propositions. The proposition 

by pA^f, is the proposition that is (rue when both p and q are tnaw aad is false 

otherwise. The proposition p A q is called the conjunction of p and q. 


The troth table for p A q is shown in Table 2, Mote that there are four rows in this 
truth table, one row for each possible combination of truth values for the propositions /> 
and u. 


Find the conjunction of the propositions p and q where p is the proposition “Today is 
Friday" and q is the proposition ‘‘It is raining today.” 

Solution: The conjunction of these propositions, p A q. is the proposition “Today is 
Friday and it is raining today.” This proposition is true on rainy Fridays and is false on 
any day that is not a Friday and on Fridays when it does not rain. ■ 

DEFINITION 3. Let p and q be propositions. The proposition “/> or q denoted 
by p\/q, is the proposition that is false when p and q arc both false and true otherwise, 
The proposition p \J q is called the disjunction of p tend q. 

The truth tabic tor p y q is shown in Table 3. 

The use of the connective or in a disjunction corresponds to one of the two ways the 
word or is used in English, namely, in an inclusive way. A disjunction is true when either 
of the two propositions in it is true or when both arc true. For instance, the inclusive or 
is being used in the statement 

“Students who have taken calculus or computer science can take this class." 

Here, we mean that students who have taken both calculus and computer science can 
take the class, as well as the students who have taken just one of the two subjects. On 
the other hand, we are using the exclusive or when we say 

“Students who have taken Calculus or computer science, but not both, can enroll 
in this class.” 

Here, we mean that students who have taken both calculus and a computer science 
course cannot take the class. Only those who have taken exactly one of the two courses 
can take the class. 


TABLE 2 The Truth 

Table for the Conjunction 
of Two Propositions, 

P 9 

P A q 

T I 

T 

T b 

F 

F T 

F 

F F 

F 


TABLE 3 The Truth 
Table for the Disjunction 
of Two Propositions. 

P 9 

p v q 

T T 

■>' 

T F 

T 

F T 

T 

. F F 

F 

_ 
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1.1 Logic 5 


EXAMPLE 5 


Similarly, when a menu at a restaurant states, “Soup or salad comes with an entree,'’ 
the restaurant almost always means that customers can have either soup or salad, but 
net both. Hence, this is an exclusive, rather than an inclusive, or. 


What is the disjunction of the propositions p and q where p and q are the same propo¬ 
sitions as in Example 4? 

Solution: The disjunction of p and q, p \, q, is the proposition 
“Today is Friday or it is raining today.” 

This proposition is true on any day that is either a Friday or a rainy day (including rainy 
Fridays). It is only false on days that are not Fridays when it also does not rain. ■ 


As wax previously remarked, the use of the connective or in a disjunction corre¬ 
sponds to one of the two ways the word or is used in English, namely, in an inclusive 
way. Thus, a disjunction is true when either of the two propositions in it is true or when 
both are true. Sometimes, we use or in an exclusive sense. When the exclusive or is 
used to connect the propositions p and a. the proposition "p or q (but not both!” is ob¬ 
tained, This proposition is true when p is true and q is false, or vice versa, and it is false 
when both p and q arc false and when both are true. 


DEFINITION 4, Let p and q be propositions, The exclusive or ofp and q, denoted 
by p © q, is the proposition that is (rue when exactly one of p and q is true and is 
false otherwise. 


The truth table for the exclusive or of two propositions is displayed in Table 4. 


We will discuss several other important ways that propositions may be combined. 

DEFINITION 5. Letp and q be propositions. The implication p ~> q is the propo¬ 
sition that is false- when p is true and q is false and true otherwise. In (his implication 
p is called the hypothesis (or antecedent or premise ) and q is called the conclusion 
(or consequence). 


The truth table for the implication p -> q is shown in Table 5, 


TABLE 4 The Truth Table 
far the Exclusive Or of Two 
Propositions, 

t> 

? 


T 

T 

F 

T 

F 

T 

F 

T 

T 

F 


F 


TABLES The Truth 
Table for the Impli¬ 
cation p —• q. 

p q 

p -> q 

T T 

T 

T F 

F 

F T 

T 

F F 

T 
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1 1 The Foundations: Logic, Sets, and Functions 


EXAMPLE 6 


Because implications arise in many places in mathematical reasoning, a wide va¬ 
riety of terminology is used to express p q. Some of the mote common ways nf 
expressing this implication are: 


■ “if p, then q 

■ "p implies q" 

■ "if A ?” 

■ “ponly ifij” 


■ “p is sufficient for if 

■ ‘f if >" 

■ “q whenever p" 

■ “q is necessary for p." 


Note that p q is false only in the case that p is true but q is false, so that it is 
true when both p and q are true, and when p is false (no matter what truth value q has), 

A useful way to remember that an implication is true when its hypothesis is false 
is to think of a contract or an obligation. If the condition specified by such a statement 
is false, no obligation is in force. For example, the statement “If you snake more than 
$25,000, then you must file a tax return” says nothing about someone snaking less than 
$25,000, You violate the obligation only if you make more than $25,000 and do not file 
a return. Similarly, the statement "If a player hits more than 60 home runs, then a bonus 
of $10 million is awarded” in the contract of a baseball player is violated only when the 
player hits more than 60 home runs, but the bonus is not awarded. This says nothing if 
the player hits fewer than 60 home runs. 

The way we have defined implications is more general than the meaning attached 
to implications in the English language. For instance, the implication 

“If it is sunny today, then we will go to the beach.” 

is an implication used in normal language, since there is a relationship between the 
hypothesis and the conclusion. Further, this implication is considered valid unless it is 
indeed sunny today, but we do not go to the beach. On the other hand, the implication 

“If today is Friday, then 2 + 3 - 5.” 

is true from the definition of implication, since its conclusion is true. (The truth value 
of the hypothesis does not matter then. ) The implication 

“If today is Friday, then 2 + 3 = 6.” 

is true every' day except Friday, even though 2 + 3 = 6 is false. 

We would not use these last two implications in natural language, since there is no 
relationship between the hypothesis and the conclusion in either implication. In math¬ 
ematical reasoning we consider implications of a more genera) sort than we use in En¬ 
glish. The mathematical concept of an implication is independent of a cause-and-effect 
relationship between hypothesis and conclusion. Our definition of an implication spec¬ 
ifies its truth values; it is not based on English usage. 

The if-then construction used in many programming languages is different from 
that used in logic. Most programming languages contain statements such as if p then 
5, where p is a proposition and S is a program segment (one or more statements to be 
executed). When execution of a program encounters such a statement, S is executed if 
p is true, but S is not executed if p is false, as illustrated in the following example 


What is the value of the variable j after the statement 
if 2 + 2 =4 then x : = x+ 1 

if a- = 0 before tins statement is encountered? (The symbol: ~ stands for assignment. 
I he statement x : = x F 1 means the assignment of the value of x + 1 to a.) 
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1.1 Logic 7 


EXAMPLE 7 


Solution: Since 2 + 2 = 4 is true, the assignment statement x : - x + 1 is executed, 
Hence, x has the value 0+1 = 1 after this statement is encountered. ■ 


We can build up compound propositions using the negation operator and the differ¬ 
ent connectives defined so far. Parentheses are used to specify the order in which the 
various logical operators in a compound proposition are applied. In particular, the logi¬ 
cal operators in the innermost parentheses are applied first. For instance, ip y q)A{-,r) 
is the conjunction of p V q and -i r. To cut down on the number of parentheses needed, 
we specify that the negation operator is applied before all other logical operators. This 
means that -i p A q is the conjunction of -i p and q, namely (-i p) A q, not the negation 
of the conjunction of p and q, namely ->(p A q). 

There are some related Implications that can be formed from p —» q. The propo¬ 
sition q —• p is called the converse of p —» q. The contrapositive of p q is the 
proposition -iq —* -i p. 


Find the converse and the contrapositive of the implication 
“If today is Thursday, then 1 have a test today.” 

Solution: The converse is 

“If 1 have a test today, then today is Thursday.” 

And the contrapositive of this implication is 

“If 1 do not have a test today, then today is not Thursday.” ■ 


We now introduce another way to combine propositions. 

DEFTNITION 6. Let p and q be propositions. The biconditional p q is the 
proposition that is true when p and q have the same truth values awl is false other¬ 
wise. 


The truth table for p ^ q is shown in Table 6. Note that the biconditional p q is 
true precisely when both the implications p ~ 1 q and q —> p are true. Because of this, 
the terminology 

~‘p if and only if q" 

is used for this biconditional. Other common ways of expressing the proposition p *-» q 
are: "p is necessary and sufficient for q" and “if p then q, and conversely,” 


TABLE 6 The Truth Table for 
the Biconditional p q. 

P 4 

p 

T T 

T 

T F 

F 

F T 

F 

F F 

I 
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I r The foundations: Logie,Sets, and Lunchfins 


TRANSLATING ENGLISH SENTENCES 


'There arc many i caserns to translate English sentences into expressions involving propo¬ 
sitional variables and logical connectives. In particular, English (and every other human 
language) is often ambiguous. Translating sentences into logical expressions removes 
the ambiguity. Note that this may involve making a set of reasonable assumptions based 
on the intended meaning of the sentence. Moreover, once we have translated sentences 
front English into logical expressions we can analyze these logical expressions to de¬ 
termine their truth values, we can manipulate them, and we can use rales of inference 
(which are, discussed in Chapter 3) to reason about them. 

To illustrate the process of translating an English sentence into a logical expression, 
consider the following examples 


EXAMPLES il nw can the following English sentence be translated into a logical expression? 

"You can access the Internet from campus only if you are a computer science 
major or you are not a freshman." 

Solution: There are many ways to translate this sentence into a logical expression. 
Although is is possible to represent the sentence by a single propositional variable, such 
as p, tins would no! be useful when analyzing its meaning or reasoning with it. Instead, 
we will use propositional variables to represent each sentence part and determine: the 
appropriate logical connectives between them, ft) particular, we let a, r, and f represent 
You can access the Internet from campus.’’ "You are a computer science major." and 
"You are a freshman.” respectively. Noting shat "only if is one way an implication can 
be expressed, this sentence can be represented as 

ci —* (c V 'i /)- 9 

EX AMPLE l > How can the following English sentence be translated into a logical expression'.' 

“You cannot tide the roller coaster if you are under 4 feet tali unless you are 
older than 16 years aid.” 

Solution: There are many ways to translate this sentence into a logical expression. The 
simplest but least useful way is simply in represent the sentence by a single ptuposi- 
tiMia! variable, say. p. Although this is not wrong, 'doing this would not assist us when 
we try io analyze the sentence or reason using it. More appropriately, what wc can do 
is to use propositional variables re represent each of the sentence parts and to decide 
on the appropriate logical connectives between them. In particular, we let q. r, and j- 
represent “You can ride the roller coaster," “You are under 4 feet tall,” and “You ait 
older than 16 years old,” respectively, Then the sentence can be translated to 
{>•/■. -m) -,q 

Of course, there are other ways to represent the original sentence as a logical ex¬ 
pression. but the one we have used should meet our needs. ■ 

BOOLEANSEARCHES 


Logical connectives are used extensively in searches of large collections of informa¬ 
nt hen, such as indexes of Web pages. Because these searches employ techniques from 
propositional logic, they are called Boolean searches. 
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In Boolean searches, the connective AND is used to match records that contain 
both of two search terms, the connective OR is used to match one or both of two search 
terms, and the connective NOT [sometimes written as AW? NOT) is used to exclude a 
particular search term. Careful planning of how logical connectives arc used is often 
required when Boolean searches are used to locate information of potential interest. The 
following example illustrates how Boolean searches tire carried out. 


EXAMPLE 10 Web Page Searching Most Web search engines support Boolean searching tech¬ 

niques. which usually can help find Web pages about particular subjects. For instance, 
using Boolean searching to find Web pages about universities in New Mexico, we can 
look "for pages matching NEW AND MEXICO AND UNIVERSITIES. The results of 
this search will include those pages that contain the three words NEW, MEXICO, and 
UNIVERSITIES. This will include all of the pages of interest, together with others 
such as a page about new universities in Mexico. Next, to find pages that deal with 
universities in New Mexico or Arizona, we can search for pages matching (NEW AND 
MEXICO OR ARIZONA) AND UNIVERSITIES. (Note: Here the AND operator takes 
precedence over the OR operator.) The results of this search will include all pages 
that contain the word UNIVERSITIES and either both the words NEW and MEXICO 
or the word ARIZONA. Again, pages besides those of interest will be listed. Finally, 
to find Web pages that deal with universities in Mexico (and nut New Mexico), we 
might first look for pages matching MEXICO AND UNIVERSITIES, but since the 
results of this search will include pages about universities in New Mexico, as well as 
universities in Mexico, it might be better to search for pages matching {MEXICO AND 
UNIVERSITIES) NOT NEW. The results of this search include pages that contain both 
the words MEXICO and UNIVERSITIES but do not contain the word NEW. ■ 


LOGIC AND BIT OPERATIONS 


Computers represent information using bits. A bit has two possible values, namely. 0 
web (zero) and 1 (one). This meaning of the word bit comes from binary digit, since zeros; and 
ones tire the digits used in binary representations of numbers. The well-known statisti¬ 
cian John Tukcy introduced this terminology in -1946. A bit can be used to represent a 
truth value, since there are two truth values, namely, true and false. As is customarily 
done, we will use a I bit to represent true and a 0 bit to represent false. That is. 1 rep¬ 
resents T (true), 0 represents F (false). A variable is called a Boolean variable if its 
value is either true or false. Consequently, a Boolean variable can be represented using 
a bit. 

Computer bit operations correspond to the logical connectives. By replacing true 
by a one and false by a zero in tile truth tables for the operators A, v, and ©, the tables 
shown in Table 7 for the corresponding bit operations are obtained. We will also use 
the notation OR, AND and X'OR for the operators \J, A, and ®. as is done in various 
programming languages. 

Information is often represented using bit strings, which are sequences of zeros 
and ones. When this is done, operations on the bit strings can be used to manipulate 
this information. 

DEFINITION 7. A hit string is a sequence of zero or more bits. The imgth of this 
string is the number of bits in the string. 
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TABLE 7 Tables for the Bit 
Operators OR, AND, and XOR. 

T H 
.r y ' 

x ■ y : x y 

.1- y 

0 0, 

0 0 

0 

0 i , i 

1 1 0 

1 

1 1 0 1 

1 0 

i ! 

1 ! ! i 

1 1 I 



EXAMPLE 11 10 IQ 10(011 is a bit string of length nine. 


We can extend bit operations to bit strings. We define the bitwise OR, bitwise AND, 
and bitwise A OR of two strings of the same length to be the strings that have as their bits 
the OR, AND, and XOR of the corresponding bits in the two strings, respectively. We 
use the symbols y*. A, and 8 to represent the bitwise OR, bitwise AND, and bitwise 
XOR operations, respectively. We illustrate bitwise operations on bit strings with the 
following e x a hi pie . 


EXAMPLE 12 Find the bitwise OR, bitwise AND, and bitwise XOR of the bit strings 01 101) 0110 

and 11 0001 1101. (Here, and throughout this book, bit strings will be split into blocks 
of four bits to make them easier to read. > 

S'olmkm: The bitwise OR. bitwise TAT), and bitwise XOR of these strings are obtained 
by taking the OR, AND, and XOR of the corresponding bits, respectively. This gives us 


Historical Note: There were several other suggested words for a binary digit, including binit and bi% xr, 
that never were widely accepted. The adoption of the word hit may be doe to its meaning as a common 
English word. For an account of Tukcy’s coining of the word bit, see the April 1984 issue of Annals oj'ih? 
H is to rv of C< rmpuiin g. 

John Wilder Ttakcy (burn 1915). Tukey, bom in New Bedford, Massachusetts, was an only child. Hi; 

Wt’b Porcius, kith teachers, decided home schooling would best develop his potential. His formal education 
began at Brown University, where he studied mathematics and chemistry. He received a master's degree 
in chemistry from Brown and continued hk studies at Princeton University, changing his field of study 
front chemistry fo mathematics. He received his Plt.D. from Princeton in 19.19 for work in topology, when 
he was appointed an instructor in jtMlJiemalks at Princeton. With the srart of World Wsi II, he joined the 
Fire Control Research Office, where he began working in statistics. Tukey found statistical research to his 
liking and impressed several leading statisticians with his skills. In 1945, at the conclusion of the war, 
Fukcy returned to the mathematics department at Princeton as a professor of statistics, and he also took a 
position at AT&T Bell Laboratories. Tukey founded the Statistics Department at Princeton in 1966 and was 
its first chairman. Tukey made significant contributions to many areas of statistics, including the analysis 
'T variance, the estimation of spectra of time series, inferences about the values afa set of parameters from 
It single experiment, and the philosophy of statistics. However, he is best known for his invention, with J. 
W. Cooley, of the fast Fourier irons Id rill, 

Tukey contributed his insight and expertise by serving on the President's Science Advisory Commit' 
tee. He chaired several important committees dealing with the environment, education, and chemicals and 
health, He also served on committees working on nuclear disarmament. Tukey has received many awards, 
including the National Medal of Science. 
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0 ! 1011 0110 
U0Q01 1101 
ii liiii mi 
01 0001 0100 
io ioio mu 


bitwise OR 
bitwise AND 
bitwise XOR 


Exercises 


1. Which of the following sentences are propositions*/ 
What are the truth values of those that are propositions? 

a) Boston is the capital of Massachusetts. 

1)1 Miami is toe capital o('Florida 

c) 2 i .1 - 5. 
d: *• • ' - 10 
el t ) 2 = li. 

f) Answer thi s quesiinn, 

g) r + v • v — x lor every pair of real numbers jr 
and v. 

2. Which nf the following are propositions? What arc the 
truth values of those that are propositions'.' 

a) Du not pass go. 

h) What time is ji ? 

ct There are no black flies in Maine. 

d) 4 + .v - 5 , 

a ;■ ! = 5 if > -- 1, 
f) i - I- v b ii t * ... 

3. What is the negation of each of the follow ing proposi¬ 
tions'* 

a’ Today is Thursday. 

St I There is no pullitiion in New Jersey, 
cl 7 r ! 3 

d) The summer in Maine is hot and sunny. 

4. Let p and q be the propositions 

p : i bought a lottery ticket this week. 
q . ! won the minion dollar jackpot on Friday. 

Express each of the foiiowmg propositions ss an En¬ 
glish sentence, 
a; r bf p\ q 

c) p '> d) p -Oq 

el p—q f) -P ~ uf 

81 ~.p -uy b) ~p\; (p A q) 

5. Lei p and q i>c the propositions 

p : It is below freezing, 
f ; ll is snowing. 

Write the following propositions using p and a and log¬ 
ical connectives. 

a) It is below freezing and snowing. 

b) it is below freezing but not snowing, 

e) It is not below freezing and it is not snowing. 


(J) It is either snowing or below freezing (or both) 
el If it is below freezing, it is also snowing, 
ft It is either below freezing or it is snowing, but it is 
not snowing if it is below freezing, 
gi That it is below freezing is necessary' and sufficient 
id: it to be snowing. 

6. Let p, q. and rbe the propositions 

p : You have the flu 
q ; You miss the final examination. 
r : You pass the course. 

Express each of the following propositions as an En¬ 
glish sentence, 
a) p -* q l») — r 

c) q ■ • .>■ d) p \i q \ r 

e) {p — -ir)Vfa ■“* -rf) 
fl ip A q) V (■" q A r j 

7. I Ait p and q be the propositions 

p : You dri ve over 65 miles per hour. 
q You get a speeding ticket. 

Write tile following propositions using p and q and log¬ 
ical connectives. 

a,i You do not drive over 65 miles per hour, 
bt YolI drive over 65 miles per hour, hut you do no! get 
a speeding ticket, 

c) You will get a speeding ticket if you drive over 
65 miles per hour. 

d) If you do not drive over 65 miles per hour, then you 
will not get a speeding ticket. 

e) Driving over 65 miles per hour is sufficient for get¬ 
ting a speeding ticket. 

f l You get a speeding ticket but you do not drive over 
65 miles per hour. 

g) Whenever you get a speeding ticket, you are driving 
over 65 miles per hour. 

8. Let p, q, and r be the propositions 

p : You get an A on the final exam. 
q : Y'ou do every exercise in this book. 
r . You get an A in this class. 

Write the following propositions using p, q, and r and 
logical connectives. 
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a) You get an A in Lhjs class, but you do not do every 
exercise in this book. 

b) You get an A on the tinal. you do every' exercise in 
this book, atui you get an A in tills class, 

c) To gel an A in this class, it is necessary for you to 
get an A on the final. 

dl You gel an A on the final, but you don't do every 
exercise in this book; nevertheless, you get an A in 
this class. 

e) Getting an A on the final and doing every exercise 
in this book is sufficient for getting an A. in this 
class. 

f} You will get an A in this class if and only if you 
either do every 1 exercise in this book or you get an 
A on die final. 

1- Determine whether cadi of the following implications 
is true or false, 

a) in + I = 2, then 2+2-5. 

b) If I + 1 - 3, then 2 + 2=4. 
ej Ifl + 1 =■ 3, then 2 + 2 = 5. 

d) If pigs can fly, then I + I - 3, 

e) If 1 + 1 - 3, then God exists, 

f) ]f | + i = t, then pigs can fly 

gi If 1 1 i : =- 2, then pigs can fly. 

h) If 2 - 2 -- 4, then 1+2-3. 

10. For each of the following sentences, determine whether 
an inclusive or or an exclusive or is intended. Explain 
your answer. 

a) Experience with C+ + or Java is required. 

b) L unch includes soup or salad. 

c) To enter the country you need a passport or a voter 
registration card. 

d) Publish or perish 

H. For each of the following sentences, state what the 
sentence means if the or is an inclusive or (that is, a 
disjunction) versus an exclusive or. Which of these 
meanings of or do you think is intended? 

a) To take discrete mathematics, you must have taken 
calculus or a course in computer science, 
bi When you buy a new car from Acme Motor Com¬ 
pany, you get $2000 back in cash or a 2% car loan. 

e) Dinner for two includes two items from column A 
or three items from column B. 

d) School is closed if more than 2 feet of snow falls or 
if the wind chill is below -100. 

12. An ancient Sicilian legend says that the barber 
in a remote town who can be reached only by 
traveling a dangerous mountain road shaves those 
people, and only those people, who do not shave them¬ 
selves. Can there be such a barber? 

13. Each inhabitant of a remote village always tells the 
truth or always lies, A villager will only give a "Yes" 
or a “No" response to a question a tourist asks. Sup¬ 
pose you are a tourist visiting this area and come to a 
fork in the road. One branch leads to the rums you want 
to visit: the other branch leads deep into the jungle. A 


villager is standing at the fork in the road. What one 
question can you ask the villager to determine which 
branch to take? 

14. An explorer is captured by a group of cannibals There 
are two types of cannibals—those who always tell 
the truth and those who always lie. The cannibals will 
barbecue the explorer unless he can determine whether 
a particular cannibal always lies or always tells 
the truth. He is allowed to ask the cannibal exactly one 
question. 

a) Explain why the question "Are yon a liar 7 " does not 
work. 

to) Find a question that the explorer can use to deter¬ 
mine whether the cannibal always lies or always 
tells the truth. 

15. Write each of the following statements in the form “if 
p, then q" in English. {Hint: Relerto the list of common 
ways to express implications listed in this section .) 

a) It snows whenever the wind blows from the north¬ 
east. 

b) The apple trees will bloom if it stays warm for a 
week. 

c) That the Pistons win the championship implies that 
they heat the Lakers. 

d) It is necessary to walk 8 miles to get to the top of 
Long's Peak, 

el To get tenure as a professor, it is sufficient to be 
world-famous. 

f) If you drive more than 4(XJ miles, you will need to 
buy gasoline. 

g) Your guarantee is good only if you bought your CD 
player less than 90 days ago. 

16. Write each of the following statements in the form “if 
P then q" in English. (Him: Refer to the list of common 
ways to express implications listed in this section.) 

a} I will remember to send you the address only if you 
send me an e-mail message. 

b) To be a citizen of this country, it is sufficient that 
you were bom in the United States. 

c) If you keep your textbook, it will be a useful, refer¬ 
ence in your future courses. 

d) The Red Wings will win the Stanley Cup if their 
goalie plays well. 

ci That you get the job implies that you had the best 
credentials. 

0 The beach erodes whenever there is a storm 

g) It is necessary to have a valid password to log on to 
the server. 

17. Write each of the following propositions in the form "p 
if and only if q' in English. 

a) It it is hot outside you buy an ice cream cone, and 
if you buy an ice cream cone it is hot outside. 

b) For you to win the contest it is necessary' and suffi¬ 
cient ihat you have the only winning ticket. 

c) You get promoted only if you have connections, and 
you have connections only if you get promoted. 
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d) If you watch television your mind will decay, and 
conversely. 

e) The trains run late on exactly those days when I take it. 
1ST Write each of the following propositions in the form “p 

if and only if q” in English, 
a) For you to get an A in this course, it is necessary 
and sufficient that you learn how to solve discrete 
mathematics problems. 

bt tf you read the newspaper every day. you will tie 
informed, and conversely. 

e) It rains if it is a weekend day, and it is a weekend 
day tf it rains. 

d) You can sec the wizard only if the wizard is not in, 
and the wizard is not in only if yon can see him. 

19. State the converse and contrapositive of each of the fol¬ 
lowing implications. 

a) If it snows today, I will ski tomorrow. 

b) 1 come to class whenever there is going to be a quiz. 

e) A positive integer is a prime only if it has no divi¬ 
sors other than 1 and itself, 

20. State the converse and contrapositive of each of the 
follovving implications. 

a) If it snows tonight, then I will stay at home. 

I» I go to the beach whenever i t is a sunny summer day. 
cl When 1 stay up late, it is necessary that I sleep until 
noon. 

21. Construct a truth table for each of the following com¬ 
pound propositions. 

a) r ■’!> b) py-ip 

el ip v i q) tj d) (p\/q) — (p /\q) 

e) ip <y) — t-n/ — -ip) 

f) (p — q) — [t{ — p) 

22. Construct a truth table for each of the following 
compound propositions. 

a) p ® p b j p @ -n p 

e) p<& iq dl ~>pG~,q 

c) (p © tj) \/ip ® ,,/} ft (p © q) A (p S 

23. Construct, a truth tabic for each of the following com¬ 
pound propositions, 

a) p - 

b) -p ->q 

e) (p — q)\'(->p q) 

d) (p -« (jj A (~p -* q) 

C) ip *--*■ rjl X'i-.p ^ q) 

f) i-ip “ 11 /1 {p «--» q) 

24. Construct a truth table for each of the following com¬ 
pound propositions. 

a) (pyq)\‘r b) (/?yy)Ar 

c) ip A q) v > d) ipi\ q) A r 

f) (pyq)^ or f) (/} f\q) y -i r 

25. Constnict a truth table for each of the following com¬ 
pound propositions. 

a) p — Irfyri 
bl -ip — (q ~ r) 

0 IP — c/Vvi-p — r) 

d) i[> ~ q) At -ip -* r) 


e) ip q)\n-iq **r) 

f) (-.p ^ -iq) {q -■> r) 

26 . Construct a tnith table for Up — 1 • 4) -* r) — 1 ■ a. 

27 . Construct a truth table for ip q) (r <-* jj. 

28 . What is the value of x after each of the following state¬ 
ments is encountered in a computer program, if x - 1 
before the statement is reached? 

a) if! - 2 = 3 then x : = x + l 

b) iff! + l = 3 ) OR (2 + 2 = 3 ) then .r 1 = j + I 

c) if (2 + 3 - 5 ) AND (3 + 4 - 7 ) then r: = a t- 1 

d) if £ 1 -r I = 2) XOR (1 + 2 = 3) then t .r + 1 

c) if x < 2 then x : - x + 1 

29 . Find the bitwise OR, bitwise AND , and bitwise XOR of 
each of the following pairs of bit strings. 

a) 101 II 10,0100001 
b| 11II 0000. 1010 1010 

c) 00 0111 0001. 10 0100 1000 

d) 1111111111, no oooo 0000 

30 . Evaluate each of the following expresMons. 

a) 1 1000 A (01011 v i 1011 ) 

b) 10 mi A i uiouyo 1000 

c ) (0 1010® 1 1011) ©0 1000 

dj t: iony 01010 ) a 1 1 0001 y i ioiij 

Fuzzy logic is used in artificial intelligence. In fuzzy logic, 
a proposition lias a truth value that is a number between 
0 and 1, inclusive, A proposition with a truth value of 0 is 
false and one with a troth value of 1 is true. Truth values that 
are between 0 and 1 indicate varying degrees of truth. For 
instance, the truth value 0,8 can be assigned to the statement 
‘Fred is happy," since Fred is happy most of the time, and 
the tnith value 0.4 can be assigned to the statement “John is 
happy," since John is happy slightly less than half the time. 

31 . The truth value of the negation of a pro post tioti in fuzzy 
logic is I minus the truth value of the proposition. 
What arc the truth values of the statements "Fred is not 
happy" and “John is not happy"? 

32 . The truth value of the conjunction of two propositions 
in fuzzy logic is the minimum of the truth values of the 
two propositions. What are the truth values of the state¬ 
ments "Fred and John are happy" and “Neither Fred nor 
John is happy’’? 

33 . The troth value of the disjunction of two propositions 
in fuzzy logic is the maximum of the truth values of 
the two propositions. What are the truth values of the 
statements "Fred is happy, or John is happy" and "Fred 
is not happy, or John is not happy"? 

* 34 . Is the assertion ‘This statement is false" a proposition? 

A set «f propositional expressions is consistent if there is 
an assignment of truth values to the variables in the expres¬ 
sions that makes each expression true. When giving sys¬ 
tem specifications it is important that these specifications 
be consistent. 

35 . Are the following specifications consistent 1 ’ "The sys¬ 
tem is in multiuser state if and only if it is operating 
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rutnttally. If lilt- system is operating normally, the ker¬ 
nel is functioning. The kernel is nor functioning or the 
system is in itttentipt mode. If the system is not in mul¬ 
tiuser state, then it is in interrupt mode. The system is 
not in interrupt mods.” 

36. Are the following specifications consistent'. 1 "If the 
tile system is not locked, then new messages will he 
queued. If the file system is nut locked, then the system 
is 1 uric I ioning normally, and con verse! v. It new mes¬ 
sages are not queued, then they will he sent to the mes¬ 
sage buffer. If the file system is not locked, then new 
messages will be sent to the message buffer, New mes¬ 
sages will not be sent ;o the message buffer" 

37. Wftal fSiiijlean search would you use to look for Web 
pages .ibiiW ocache, j r i .New Jersey 1 ’ What if you 
wonted to .find Web pages about benches on the isle ,0' 
Jersey tin the Fnp'udi rherirarfr.' 

38. What Boolean search would you use to look for Web 
pages: about hiking in West Virginia'? What if you 
wanted to rind Web pages about hiking in Virginia, but 
mu u; West Virginia' 1 

Exercise:. hi-A? ,r< piiiczks thdt can be solved by translat¬ 
ing statements into logical expressions and reasoning from 

these cKpft\Moii> using truth table. 1 ;. 

3?. Steve would like w ikumiiitc the nsJalivc salaries at' 
three ,v workers using two facts, first, he knows that if 
Fred nut the highest paid of the three, then Janice is. 
.Second he knows that it Janice is not the lowest paid, 
the ■ Muggi.i: is paid the most. K it passible w determine 


the relative .salaries of Fred, Maggie, and Janice from 
what Steve knows? If so, who is paid the most and who 
Lhe least? Explain your reasoning. 

40. Five friends have access to a chat room. Is it possible to 
determine who is chatting if the following information 
is known? Either Kevin or Heather, or both, are chat¬ 
ting. Either Randy or Vijay. but not both, are chatting. 
If Abby is chatting, so is Randy, Vijay and Kevin are 
either both chatting or neither is. If Heather is chatting, 
then 50 are Abby and Kevin. Explain your reasoning. 

41. A detective has interviewed four witnesses to a crime, 
From the stories of the witnesses the detective has con¬ 
cluded that it the butler is idling the truth then so is the 
cook; the cook and the gardener cannot both be telling 
rhe truth; the gardener and the handyman are not both 
lying; and it' the handyman is telling the truth then the 
cook ut lying. For each of the four witnesses, can the 
detective determine whether that person is telling the 
Truth or lying? Explain your reasoning. 

42. Four friends have been identified as suspects for an 
unauiliori/ed access info a computer system. They have 
made statements to the investigating authorities. Al¬ 
ice said "Carlos did it.” John said “1 did not tin it." 
Carlos said "Diana did it.” Diana said "Carlos lied 
when he said that I did it.” 

ai if (he authorities also know that exactly one of the 
four.suspects is idling the truth, who did id Explain 
your reasoning. 

1*1 If the authorities also know that exactly one is lying, 
who did it? Explain your reasoning. 


Propositional Equivalences 


INTRODUCTION 


An important type of step used in u mathematical argument is the replacement of a 
statement with another statement with the same truth value. Because of this, methods 
ihat produce propositions with the same truth value as a given corn pound proposition 
arc used extensively m the cmf'trueiidit of mathematical arguments, 

We ocean our discussion with a classiiication of compound propositions according 
to then' possib;,; Truth values,. 


DEFINITION 1, A compound proposition that is always true, no matter what the 
truth values of the propositions that occur in it, is called a tautology, A compound 
proposition that is always false is called a contradiction. Finally, a proposition that 
ts iin(her a tautology nor a contradiction is called a contingency. 

Tautologies and contradict ions are often important in inaihemalical reasoning. The 
following example illustrates these types of propositions. 
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EXAMPLE 1 


EXAMPLE 2 


TABLE 1 Examples of a Tautology 
and a Contradiction. 

P 

~*P 

PV-ip 

p A -./> 

T 

F 

T 

E 

F 

T 

. 

T 

F 


We can construct examples of tautologies and contradictions using just one proposition. 
Consider the truth tables of p y -i p and p A -~,p t shown in Table 1. Since p y -i p is 
always true, it is a tautology. Since p/\~,p is always false, it is a contradiction ■ 


LOGICAL EQUIVALENCES 


Compound propositions that have the same truth values in all possible cases arc called 
logically equivalent. We can also define this notion as follows. 


DEFINITION Z The propositions p and q are called tegicaliy equivalent 
if p q is a tautology. The notation p q denotes that p and q are logically 
equivalent. 


One way to determine whether two propositions are equivalent is to use a truth 
table. In particular, the propositions p and q are equivalent if and only if' the columns 
giving their truth values agree. The following example illustrates this method. 


Show that -ifp y q) and -ip A -u/ are logically equivalent. This equivalence is one of 
De Morgan's laws for propositions, named after the English mathematician Augustus 
De Morgan, of the mid-nineteenth century. 

Solution: The truth tables for these propositions are displayed in Table 2. Since the truth 
values of the propositions -i (p y q) and -i p A -i q agree for all possible combinations 
of the truth values of p and q, it follows that these propositions are logically equivalent. 


TABLE 2 Truth Tables for -i (p \J qt ant: 

-P r 

-i q- 

p q 

pjq 

•P '/' 

IA7T 


->p A ~<q 

T T 

T 

F 

F 

F 

F 

T F 

T 

F 

F | 

T 

F 

F T 

T 

F 

T I 

F 

F 

lL_L_ 

h 

T 

T ! 

r 

T 
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EXAMPLE 3 


EX AMPLE 4 


TABLE 3 

Troth Tables for -p y q and p — q. 

P 

<I 

1 

' ~P 

“> pVq 


T 

T 

F 

T 

T 

r 

F 

F 

P 


F 

T 

1 T 

T 

T 

K 

F 

! T 

T 

T 


Show shat the propositions p q and m p y q are logically equivalenL 

Solutiou: We construct (he truth table for these propositions in Table 3, Since the truth 
values of -/) \/ q and p q agree, these propositions are logically equivalent. ■ 

Show that the propositions p y U] A r) and ( p y q) A (p\; r) are logically equivalent. 
This is the distributive law of disjunction over conjunction. 

Solution: We construct the truth table for these propositions in Table 4. Since the truth 
values of p y (q A r) and (p y q) A (p y r) agree, these propositions are logically 
equivalent. ■ 


Remark; A truth table of a compound proposition involving three different proposi¬ 
tions requires eight rows, one for each possible combination of truth values of the three 
propositions. In general. 2 n rows are required if a compound proposition involves « 
propositions. 


Augustus De Morgan ( 1806 - 1871 ). Augustus De Morgan was bom in India, where his father was a 
U €t «))wie1 in the Indian army. De Morgan's family moved to England when he was 7 months old, Ile attended 
private schools, where he developed a strong interest in mathematics in his early teens. De Morgan studied 
at Trinity College, Cambridge, graduating in 1827. Although he considered entering medicine or law, he 
decided on a career in mathematics, He won a position at University College, London, in 1828. but resigned 
when the college dismissed a fellow professor without giving romona. However, he resumed this position 
in 1836 when his successor died, staying there until 1866. 

De Morgan was a noted teacher who stressed principles over techniques. His students included many 
famous mathematicians, including Ada Augusta, Countess of Lovelace, who wax Charles Babbage's col¬ 
laborator iti his work on computing machines (see page 16 for biographical notes on Ada Augusta). (De 
Morgan cautioned the countess against studying loo much mathematics, since it migtiL interfere with her 
childbearing abilities' i 

De Morgan was an extremely prolific writer. He wrote more than 1000 articles for more than 15 pe¬ 
riodicals. De Morgan also wrote textbooks on many subjects, including logic, probability, calculus, and 
algebra. In 1S?8 he presented what was perhaps the first dear explanation of an important proof technique 
known as mathematical induction (discussed in Section 3,2 of this text), a term he coined. In the 1840s De 
Morgan made fundamental contributions to the development of symbolic logic. He invented notations that 
helped him prove propositional equivalences, such as the laws that are named after him. In 1842 De Morgan 
presented what was perhaps the first precise definition of a limit and developed some tests for convergent, 
of infinite series. De Morgan was also interested in the history of mathematics and wrote biographies of 
Newton and Haltev. 

In 1837 De Morgan married Sophia Frend, who wrote his biography in 1882. De Morgan's research, 
writing, and teaching left little time for his family or social life. Nevertheless, he was noted for his kindness, 
humor, and wide range of knowledge. 
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TA BLE 4 A Demonstration That p y (5 A r) and (p 
Equivalent. 

V q ) A (p y r) Are Logically 

P V 

T 

q A r 

p\/(q,\r) 

P\/Q 

P V r 

(p V q) A (p y r) 

T T 

T 

T 

T 

T 

T 

T 

T T 

F 

F 

T 

T 

T 

T 

T F 

T 

F 

T 

T 

T 

T 

T F 

F 

F 

T 

T 

T 

T 

F T 

T 

T 

T 

T 

T 

T 

F T 

F 

F 

F 

T 

F 

F 

F F 

T 

F 

F 

F 

T 

F 

F F 

F 

F 

_ 

F 

F 

F 

_ 

F 


Table 5 contains some important equivalences.* In these equivalences, T denotes 
any proposition that is always true and F denotes any proposition that is always false. 
The reader is asked to verify these equivalences in the exercises at the end of the section. 

The associative law for disjunction shows that the expression p y q y r is well 
defined, in the sense that it does not matter whether we first take the disjunction of p and 


TABLE S Logical Equivalences. 

Equivalence 

Atwne 

p AT < '• p 

P V F <==> p 

Identity laws 

i 

p y I - I 

p A F - :• F 

Domination laws 

P V p p 

P A p c=> p 

Idempotem laws 

p 

Double negation law 

r . q <rT "' q 'vp 

p ‘ Q q , 

Commutative laws 

(pyq)V'r <=^ pV(q\/r) 

{p A q) A r p A iq A r) 

Associati ve laws 

p V Uj A r) <^> (p 'yq}A{p\/ r) 
p A{q\jr) *:z=> ipAq)y(pAr) 

Distributive laws 

Ap A i ]) p y -uji 

■•(p \/q) p A -■ ij | 

De Morgan's laws 


♦These identities are a special case of identities that hold for any Boolean algebra Compare them with jet 
identities in Table I in Section 1.5 and with Boolean identities in Table 5 in Section 9. ]. 
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EXAMPLE 5 


EXAMPLE ft 


TABLE 6 Some Useful 
Logical Equivalences. 

p \l ^ p T 
p A -i p «AA> F 
(P- q) a=a (ipVffJ 


q and then the disjunction of p\/q with r, or if we first take the disjunction or q arid r and 
then take the disjunction ofp and q\/r Similarly, the expression pAqAr is wdl defined. 
By extending this reasoning, it follows that p\ y p 2 y - • - V P» and p\ A pj A ■ • ■ A p, ; 

are well defined whenever p i, p 2 . p„ are propositions. Furthermore, note that De 

Morgan’s laws extend to 

“dPl '■-/ Pi ‘' ’ V Pn) <=> A -,p> A ■ ■ ■ A - ,*„) 

and 

->(P i Ap 3 --*Ap B j y -■p 2 v •'* V -’M 

(Methods for proving these identities will be given in Chapter 3.) 

The logical equivalences in Table 5, as wdl as any others that have been estab¬ 
lished (such as those shown in Table 6), can be used to construct additional logical 
equivalences. The reason tor this is that a proposition in a compound proposition can 
be replaced by one that is logically equivalent to it without changing the truth value 
of the compound proposition. This technique is illustrated in Examples 5 and 6, where 
we also use the fact that if p and q are logically equivalent and q and r are log ically 
equivalent, then p and r are logically equivalent (see Exercise 40). 


Show that -i (p y (■ i p A q)) and -i p A ~>q are logically equivalent. 


Solution: We could use a truth table to show that these compound propositions are 
equivalent. Instead, we will establish this equivalence by developing a series of logical 
equivalences, using one of the equivalences in Table 5 at a time, stoning with <(p y 
t i p A q)) and ending with - p A -iq. We have the following equivalences, 


1 (P V (-i P A q)) <=> -i p A -i ( —i p A q) 

■AA> />A<; •; pry q\ 
c=> -i/>A(/>y-,q) 

<=> (p A p) y (~i p A -1 q'l 
F . ' -P ‘ 
f-ipA F 

'“A p A ■ i q 


from the second De Morgan's law 
from the hi st De Morgan’s mw 
from the double negation law 
from the distributive law 
since a p A p ? E 
from the law for >.lispineliur- 
!rom the identity law for E 


Consequently ->(py (i/» A q)) and -p A w/ are logically equivalent. 


Show that <p/\q) — ip if: is a tautology. 

Solution: To show that this statement is a tautology, we will use logical equivalences 
to demonstrate that it is logically equivalent to T (A kite: This could also he done using 
a truth table.) 
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ip A <ri 


U> V if) 


;/•' . </i 

■ y 'P q\ 

( -, p p} y ( -; £/ y (/ ) 


TVT 


c=> x 


by Example 2 

by the first Do Morgan's law 
hy the associative and commuta¬ 
tive laws tor disjunction 
by Example 1 and the commuta¬ 
tive law' for disjunction 
by the domination law 9 


A truth table can be used to determine whether a compound proposition is a tau ¬ 
tology. This can be done by hand for a proposition with a small number of variables, 
but when the number of variables grows, this becomes impractical. For instance, there 
are 2 J(i = 1.048,576 rows in the truth value table for a proposition with 20 variables 
Clearly, you need a computer to help you determine, in this way, whether a compound 
proposition in 20 variable ;s> a tauiology. But when there are 1000 variables, can even 
a computer determine in a reasonable amount of time whether a compound proposition 
is a tautology? Checking every one of the 2 1000 (a number with more than 200 decimal 
digits) possible combinations of truth values simply cannot be done by a computer in 
even trillions of years. Furthermore, no other procedures are know n that a computer can 
follow to determine in a reasonable amount of time whether a compound proposition in 
such a large number of variables is a tautology. We will .study questions such as this in 
Chapter 2, when wc study the complexity of algorithms. 


Exercises 


1. Use truth tables to verify the following equivalences, 

a) p A T c=r* p b) p V <?_■ > p 

C) n i -rims ¥ d> i) v T V—T 

e) i>y p <=* p I) p- -p <=■-'' f 

I. Show that -■( —iyr) and p are logically equivalent. 

J. Use truth tables rn verify the commutative laws 

a) p\'q q y P 

b) p A. q <=—> q A p 

4. Use iruth tables to verify the associative laws 
il) < P \ ■ q) V r C~.ro p y j t j ■: / r ) 

bp (/> ■■' qi A *■ coo p/\(q ,\r) 

5 . Use iruth to verify the distributive law p A 

Uf\ m -■“> vApf'-'-i. 


6. Use a truth table to verity the equivalence -i 

{/> A. t f) <_-.m -,p j -up 

7. Show that each of the following implications is a tau¬ 
tology by using truth tables. 

aj [p>\q) ■— p h) p — ip y q) 

e) -i p - \p — q) d) [p Ac/) — [p — q\ 

e) -■yp — q) — p f) ■'(;) — q) -* -<q 

8. Show- that each of the following implications is a tau¬ 
tology by using truth tables, 

;U \-p'Apyqy\ q 

h) [(/’ — i/) A if/ -* iif - ip — r) 

ft [/.’ ip -* t/ij -- q 

dj lip v't/5 <Mp ~ rjAlq -p- rj| -■ .> 


Vda Augusta, Countess of l.uveiace 1 1H15-1852 1 , Ada Augusta was the only child from the marriage 
nrff t>f the famous poet land Byron and Annahelia Millbankc, who separated when Ada was 1 month old. She 
V.us raised by her mol her. who encouraged her intellectual talents. Stic was taught by the mathematicians 
Wiliiam trend and Augustus De Morgan. In 1838 -.be man ted 1 .nrd King, later elevated to Earl of Lovelace. 
Together they had three children. 

Asia Augusta continued her mathematical studies after her marriage, assisting Charles Babbage in 
his work on an early ttMtpultng machine, cubed shii Analytic Engine. The most complete accounts of this 
machine are Ioijjiu in her writings. Alter IK-rh she and Babbage worked inward the development ot a system 
to predict horse rates. Unfortunately, their system did not work well leaving Ada heavily in debt at the time 
■d her death. The programming language Ada is named in honor of the Countess of Lovelace. 


PDF created with pdfFactory trial version www.pdffactorv.com 






20 1 / The Foundations: Logic, Seta, and Functions 


9. Show that each implication in Exercise 7 is a tautology 
without using truth tables. 

UK Show that each implication in Exercise 8 is a tautology 
without using truth tables. 

11. Verily the following equivalences, which arc known as 

the absorption laws. 

a) \/>\npAq)] < > p 

h.) In •' •• p 

12 . Determine whether i-ip A(p ijfj) -iq is a tau¬ 
tology. 

13. Determine whether I -.q A (p —> q)) — -p js a tau¬ 
tology, 

14. Show that p -* q and (p A q )y {-ip A - q) are equiv¬ 
alent. 

15. Show that ip - - q) -• r and p —*■ try r) are not 
equivalent, 

16. Show that p ■-*■ q and -iq —» -i p are logically equiv¬ 
alent. 

17. Show that - p '-*■ q and p -i q are logically equiv¬ 

alent. 

18. Show that - (jj® q) and p q are logically equiva¬ 
lent. 

19. Show that -i(/i ~ q) and -ip •-> q are logically 
equivalent. 

The dual Ufa compound proposition that contains only the 

logical operators y. A, and n is the proposition obtained 

hy replacing each y by A, each A by y. each T by F. and 

each F by T The dual of proposition s is denoted by s‘. 

20. Find the dual of each of the following propositions, 

al pA-u/A-if b) (ytAqArjy.v 

Cl ip v FI A {q'y T) 

21. Show that (TV = j. 

22. Show that the logical equivalences in Table 5, except 
far the double negation law, come in pairs, where each 
pair contains propositions that are duals of each other. 

**23. Why arc the duals of two equivalent compound propo¬ 
sitions also equivalent, where these compound propo¬ 
sitions contain only the operators A, y, and -i ? 

24. Find a compound proposition involving the proposi¬ 
tions />, q. and r that is: true when p and q are true and t 
is false, but is false otherwise. [Hint: Use a conjunction 
of each proposition or its negation.) 

25. Find a compound proposition involving the proposi¬ 
tions p, q, and r that is true when exactly two of p, q, 
and r are true and is false otherwise. [Hint: Form a 
disjunction of conjunctions. Include a conjunction for 
each combination of values for which the proposition is 
true. Each conjunction should include each of the three 
propositions or their negations.) 

26. Suppose that a truth table in n propositional variables 
is specilied. Show that a compound proposition with 
this truth table can be formed by taking the disjunc¬ 
tion of conjunctions of the variables or their negations, 
with one conjunction included for each combination of 
values for which the compound proposition is Irue. The 


resulting compound proposition is said to be in dis¬ 
junctive normal form. 

A collection of logical operators is called functionally com¬ 
plete if every compound proposition is logically equivalent 
to a compound proposition involving only these logical op¬ 
erators. 

27. Show that i, A, and y form a functionally complete 
collection of logical operators. (Hint: Use the fact that 
every proposition is logically equivalent (o one in dis¬ 
junctive normal form, as shown in Exercise 26.) 

*28, Show that -i and As form a functionally complete collec¬ 
tion of logical operators, (Hint: First use De Morgan's 
law' to show that p y q is equi valent to-il-ip A q ).) 
*29. Show that -i and y form a functionally complete col¬ 
lection of logical operators. 

The following exercises involve the logical operators NAND 
and NOR. The proposition.rVA NDq is true when either/) or 
q, or both, are false; and it i s false w'hen both p and q are true. 
The proposition p NOR q is true when both p and q are false, 
and it is false otherwise. The propositions p NANO q and p 
NOR q arc denoted by p \ q and p j q, respectively. (The op¬ 
erators | and j are called the ShelTer stroke and the Peirce 
arrow after H. M She tier and C. S. Peirce, respectively.) 

30. Construct a truth table for the logical operator HAND. 

31. Show that p j q is logically equivalent to (p A q), 

32. Construct a truth table for the logical operator NOR. 

33. Show' that p j q is logically equivalent to -dp y ql 

34. In this exercise wc will show that {/} is a functionally 
complete collection of logical operators, 

a) Show that p j p is logically equivalent to -i p, 

b) Show that (/.) j q) j. (p j q) is logically equivalent 
to p\jq. 

cj Conclude from parts (a) anti (b), and Exercise 29. 
that {|} is a functionally complete collection of log¬ 
ical operators. 

*35. Find a proposition equivalent to p —• q using only [he 
logical operator J.. 

36. Show that {;} is a functionally complete collection of 
logical operators, 

37. Show that p \ q and q I p are equivalent. 

38. Show that p \ [q \ r) and ip \ q) | r are nol equivalent, 
so that the logical operator | is not associative. 

*39. How many di fferent truth tables of compound proposi¬ 
tions are there that involve the propositions p and ql 

40. Show that if p, q, and r are compound propositions such 
that/» and q are logically equivalent and q and r are log¬ 
ically equivalent, then p and rare logically equivalent 

41. 'ITic following sentence is taken from the specification of 
a telephone system: "If the directory data base is opened, 
then the monitor is put in a closed state, if the system 
is not in its initial state.’’ This specification is hard to 
understand since it involves two implications. Find an 
equivalent, easier-to-understand specification that in¬ 
volves disjunctions and negations but not implications. 
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L3___ 

Predicates and Quantifiers 

INTRODUCTION 

Statements involving variables, such as 

“a > 3," ",r = y 4- 3.” and u .r f v - z," 

are often found in mathematical assertions and in computer programs. These statements 
are neither true nor false when the values of the variables are not specified. In this sec¬ 
tion we will discuss the ways that propositions can be produced from such statements. 

The statement ‘"vis greater than 3” has two parts. The first part, the variable x, i s the 
subject of (he statement. The second part—the predicate, “is greater than 3”—refers to 
a property that the subject of the statement can have. We can denote the statement "x is 
greater than 3“ by P(x\ where P denotes the predicate “is greater than 3” and x is the 
variable. The statement P(x) is also said to be the value of the propositional function 
P at x. Once a value has been assigned to the variable x, the statement P(x) becomes a 
proposition and has a truth value. Consider the following example. 


EXAMPLE 1 Eel P[x) denote the statement “x > 3,” What are the truth values of Pi 4) and /‘(2)7 


Charles Sanders Peirce (1839-1914). Many consider Charles Peirce the most original and versatile m- 
W€b lellet:t from , * le ' r| ire;l States; he was horn in Cambridge. Massachusetts. His lather, Benjamin Peirce. was 
a professor of mathematics and natural philosophy at Harvard. Peirce attended Harvard (1855-1859) and 
received a Harvard master of arts degree (1862) and an advanced degree in chemistry from the Lawrence 
Scientific School (1863). His father encouraged him to pursue a career in science, bin instead he chose to 
study logic and scientific methodology. 

In 1861, Peirce became an aide in the United States Coast Survey, with the goal of better understanding 
scientific methodology. His sendee for the Survey exempted him from military service during the Civil War. 
While working for the Survey, Peirce carried out astronomical and geodesic work. He made fundamental 
contributions to the design of pendulums and to map projections, applying new mathematical developments 
in the theory of elliptic functions. He was the first person to use the wavelength of light as a unit of mea¬ 
surement. Peirce rose to the position of Assistant for the Survey, a position he held until he was forced to 
resign in 1891 w hen he disagreed with the direction taken by the Survey’s new administration, 

Although making his living from work in the physical sciences, Peirce developed a hierarchy of sci¬ 
ences. with mathematics at the top rung, in which the methods of one science could he adapted for use by 
those sciences under it in the heirarchy. He w as also the founder of the American philosophical theory of 
pragmatism. 

The only academic position Peirce ever held was as a lecturer in logic at Johns Hopkins Universitv in 
Baltimore from 1879 to 1884, His mathematical work during this lime included contributions to logic, set 
theory, abstract algebra, andthe philosophy of mathematics. His work is still relevant today; some of his work 
on logic has been recently applied to artificial intelligence. Peirce believed that the study of mathematics 
could develop the mind's powers of imagination, abstraction, and generalization. H is diverse activities after 
retiring from the Survey included writing fur newspapers and journals, contributing to scholarly dictionaries, 
translating scientific papers, guest lecturing, and textbook writing, Unfortunately, the income from these 
pursuits was insufficient to: protect him and his second wife from abject poverty, He was supported in his 
1 filer years by a fund created by his many admirers and administered by the philosopher William James, 
his lifelong friend, Although Peirce wrote and published voluminously in a vast range of subjects, he left 
more than 100.000 pages of unpublished manuscripts. Because of the difficulty of studying his unpublished 
writings, scholars have only recently started to understand some of his varied contributions. A group of 
people is devoted to making his work available over the Internet to bring it better appreciation of Peirce’s 
accomplishments to the world. 
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EXAMPLE 2 


EXAMPLE 3 


EX/WIl’Lfcl 4 


Solution: The statement P(4) is obtained by setting a = 4 in the statement "x > 3." 
Hence, P34). which is the statement “4 > 3,” is true. However, P(2). which is the state¬ 
ment “2 > 3,"’ is false. ■ 


We can also have statements that involve more than one variable. For instance, 
consider the statement “a - v + 3.” We can denote this statement by Q(x, y), where x 
and y tire variables and Q is the predicate. When values are assigned to the variables x 
and y. the statement Q(x, y) has a truth value. 


Let Q(x, y) denote the statement "x - y 4 3.” What are the truth values of the propo¬ 
sitions (XL 2) and Q( 3,0)? 

Solution: To obtain Q( l, 2), set x ■= 1 and y = 2 in the statement Q(x, yi. Hence, 
Q{1, 2) is the statement "1 =* 24 3,” which is false. The statement Q(X 0) is the propo¬ 
sition “3 — 0 + 3,"’ which is true. ■ 


Similarly, we can let R(x, y, z) denote the statement "v -!- y - -T When values are 
assigned to the variables .r, y, and this statement has a truth value. 


What are the truth values of the propositions R( 1.2, 3j and R( 0,0. 1)? 

Solution: The proposition R( 1,2,3) is obtained by setting x ~ 1, v = 2, and t. - 3 in 
the statement R(x, y, i). We see that a( 1, 2, 3) is the statement “1 4 2 = 33’ which is 
true. Also note that M3 0, 1), which is the statement *'0 t- 0 - 1is false. ■ 


in general, a statement involving the n variables i , c..,t„ can be denoted by 

P(xi. x 2 , .v«). 

A statement of the form Fix., x 2 . .v„) is the value of the propositional function P 

at the n-tuple U|, x 2 . x„ }, and P is also called a predicate. 

Propositional functions occur in computer programs, as the following example 
demonstrates. 


Consider the statement 

if x > 0 then x : ^ ,t+ L 

When this statement is encountered in a program, the value of the variable a at that 
point in the execution of the program is inserted into P(x). which is “,v > 03' If P( v) is 
true for this value of a, the assignment statement ,v : ■- x - i is executed, so the value 
oi x is increased by 1. if / J (x> is false for this value of x. the assignment statement is 
not executed, so the value of .v is not changed. ■ 
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EXAMPLE 5 


QUANTIFIERS 


When all the variables in a propositional function are assigned values, the resulting 
statement has a truth value. However, there is another important way, called quantifi¬ 
cation, to create a proposition from a propositional function. Two types of quantification 
will be discussed here, namely, universal quantification and existential quantification. 

Many mathematical statements assert that a property is true for all values of a vari¬ 
able in a particular domain, called the universe of discourse. Such a statement is ex¬ 
pressed using a universal quantification. The universal quantification of a propositional 
function is the proposition that asserts that P(x) is true for ail values of a in the uni¬ 
verse of discourse. The universe of discourse specifies the possible values of the vari¬ 
able v. 

DEFINITION 1 . The wimrml quantification of Fix) is the proposition 
“P(x) is true for all values ofx in the universe of discourse." 


The notation 
V s P{x) 

denotes the universal quantification of P(x). Here V is called the universal quantifier. 
The proposition Vx P(x) is also expressed as 

"for all x P(x )” or "for every x P(xl" 

Remark: it is best to avoid the word ■‘any" since it is often ambiguous as to whether 
it means "every” or “some.” In some cases, "any" is unambiguous, such as when it is 
used in negatives, for example, "there is not any reason not. to study hard" 


Express the statement 

"Every student in this class has studied calculus" 
as a universal quantification. 

Solution: Let /*(.r) denote the statement 
'*.» has studied calculus.' 

Then the statement “Every student in this class has studied calculus” can be written as 
Va P(x), where the universe of discourse consists of the students in this class. 

This statement can also be expressed as 

Va '.Vi a) — /’ (x» 
where S{x) is the statement 
"a is in this class.’’ 

P( r) is as before, and the univ erse of discourse is the set of all students. ■ 


Example 5 illustrates that there is often more than one good way to express a quan¬ 
tification. 


PDF created with pdfFactory trial version www.pdffactorv.com 




1 / The Foundations; Logic, Sets, and Functions 


EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


Let P(x t he the statement "j f t > ,t.” What is the truth value of the quantification 
W.v Ax). where the universe of discourse is the set of real numbers? 

Solution ■ Since P( x) is true for all real numbers x. the quantification 
V.V /fovj 

is true. b 


Let £>(x) be the statement “x < 2.” What is the truth value of the quantification V.r (.La). 
where the universe of discourse is the set of real numbers' 7 

Solution: Q(x) is not true for all real numbers a, since, for instance, QO) is false, 't hus 
Va Q[x\ 

is false. ■ 


When all of the elements in the universe of discourse can be listed—say, .q, . 

■ l 'n— i it follows that the universal quantification Va Ax) is the same as the conjunction 

/for,) A A • • • A P(x a ). 

since this conjunction is true if and only if Ax,), P(x 2 ) ./for,,) are all true. 


What is the truth value of Via/ for), where Fix) is the statement "at 2 < 10” and the 
universe of discourse consists of the positive integers not exceeding 47 

Solution: The statement V.r P[x) is the same as the conjunction 
A 1) A P(2i A A3) A Pil). 

since the universe of discourse consists of the integers 1,2. 3, and 4. Since A4j, which 
is the statement “4" < ID," is false, it follows that V.r Fix) is false. ■ 


Many mathematical statements assert that there is an element with a certain prop¬ 
erty. Such statements arc expressed using existential quantification. With existential 
quantification, we form a proposition that is true if and only if P(x) is true for at least 
one value of x in the universe of discourse. 

DEFINITION 2. The existential quantification of F(x) is the proposition 
'There exists an element x in the universe of discourse such that P(x) is true.” 

We use the notation 
3 a- /for) 

for l he existential quantification ot Ax), Here 3 is called the existential quantifier. 
The existential quantification 3.rP(,t) is also expressed as 

“There is an x such that As:.' 

“There is at least one a such that /for),” 
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EXAMPLE 9 


EXAMPLE 10 


EXAMPLE 11 


or 


"For some x Pyx)," 


Let Pix) denote the statement “x > .3.” What is the truth value of the quantification 
3a Fix), where the universe of discourse is the set of real numbers? 

Solution: Since “.r > 3” is true—for instance, when x - 4 —the existential quantifi¬ 
cation of P(x), which is 3 v /*(*), is true. ■ 

Let Q{x) denote the statement “x = x + 1What is the truth value of the quantification 
3x Q< a ), where the universe of discourse is the set of real numbers? 

Solution: Since Q(x) is false for every real number x. the existential quantification of 
which is 3x Qix), is false. ■ 


When all of the elements in the universe of discourse can be listed—say, jq, x : . 

x „—the existential quantification 3,v P(x) is the same as the disjunction 

/V-,;) . Fi.X,) . • 

since this disjunction is tme if and only if at least one of P(x\), P{xP),.... P(.x„) is true. 


What is the truth value of 3xP(.v) where Pix) is the statement ''jr > 10” and the 
universe of discourse consists of the positive integers not exceeding 4? 

Solution: Since the universe of discourse is -{1,2.3,4}, the proposition 3 x Pix) is the 
same as the disjunction 

P{\)yP{2) vraivw 

Since P(4), which is the statement **4 2 > 10,” is true, it follows that 3x P(.t) is true.B 


Table I summarizes the meaning of the universal and the existential quantifiers. 

It is sometimes helpful to think in terms oflooping and searching when determining 
the truth value of a quantification. Suppose that there arc n objects in the universe of 
discourse for the variable x. To determine whether Vx Fix) is true, we can loop through 
all n values of x to see if Fix) is always true. If we encounter a value ,v for which Pix) 
is false, then we have shown that V_i P(x) is false. Otherwise, Vx P(,t) is true. To see 


whether 3.v Pix) is true, we, loop through the n 

values of x searching for a value for 

TABPEJ 

Quantifiers, 


Statement 

When True? 1 

When False? 

Vx pyx) 

3 1 Pix) 

, Pix) is true for every x, j 

j There is an x for which / J (.r) is true, j 

There is an x. for which Pix) is false, 
P(x) is false for every x. 
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which Fix) is true. If we find one, then 3.x P(x) is true If we never find such an x, 
we have determined that 3xP(x) is false. (Note that this searching procedure does not 
apply if there are infinitely many values in the universe of discourse. However, it is still 
a useful way of thinking about the truth values of quantifications.) 

Sometimes expressions involving quantifiers can be quite complicated. Translating 
a complicated expression into English helps understanding of its meaning. The first .step 
in this translation is to write out what each quantifier means. The next step is to express 
this meaning in a simpler sentence, Consider the following examples. 


EXAMPLE 12 Translate the statement 

V.r(CU') V 3 v(C( y) A Fix, y)H 

into English, where C(.r) is "x has a computer." F(i,y) is ”.t and y are friends,” and 
the universe of discourse for both x and y is the set. of all students m your school 

Solution: The statement sacs that for every student jr In your school x lias a computer 
or there is a student y such that y has a computer and.«and y are friends, In other words, 
every student in your school hits a computer or has a friend who has a computer. ■ 


EXAMPLE 13 Translate the statement 

a.rVvVriTFfx. vtAFUAAc * j) — it tv.."". 

into English, where F{tt,k) means u and /rare friends and the universe of discourse for 
i. y. and : is the set of all students in your school. 

Solution. This statemcm says that there is a student a *uch that for all students \ and 
all students r other than y, if v and y are friends and x and z arc friends, then y and 
are not friends. In other words, there is a student none of whose friends are also friends 
with ouch other. ■ 


Complicated expressions involving quantifiers also arise in mathematical state¬ 
ments. This is illustrated in the following exarnple. 


KXAMPLL 14 Assume that the universe of discourse lor the variables x and y is the set of ail real 

numbers. The statement 

VaViA -t-y - + .u 

says that v +■ y --- y F x for all real numbers ,c and y. This is the commutative Jaw for 
add;Lion of real numbers. Likewise, the stalenicnl 

Jri ip + y = 0) 

says that for every real number a then is a real number y such that ,ttv- 0. This 
states that every real number has an additive inverse. Similarly, the statement 

V.vVyVz (jc * fy + z) - (x + yi 4 z) 

is the associative law for addition of real numbed:. ■ 
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TRANSLATING SENTENCES INTO LOGICAL EXPRESSIONS 

In Section LI we illustrated the process of translating English sentences into logical ex¬ 
pressions involving propositions and logical connectives. Now that we have discussed 
quantifiers, we can express a wider variety of English sentences using logical expres¬ 
sions. Doing so eliminates ambiguity and makes it possible to reason with these sen¬ 
tences. (.Section 3.1 covers rules of inference for reasoning with logical expressions.) 

The following examples show how to use logical operators and quantifiers to ex¬ 
press English sentences, similar to the kind that occur frequently in mathematical state¬ 
ments, in logic programming, and in artificial intelligence. 

EXAMPLE 15 

Express the statements “Some student in this class has visited Mexico" and “Even,' 
student in this class has visited either Canada or Mexico" using quantifiers. 

Solution: Let the universe of discourse for the variable x be the set of students in your 
class. Let M{x) be the statement “x has visited Mexico" and C(x) the statement “x has 
visited Canada." The statement “Some student in this class has visited Mexico" can be 
w ritten as 3xM (x). The statement “E very student in this class has visited either Canada 
or Mexico” can be written as V r(C( u , v M(x )) (assuming that the inclusive, rather than 
the exclusive, or is what is meant here). ■ 

EXAMPLE 16 

Express the statement “Every one has exactly one best friend’ 1 as a logical expression. 

Solution: Let Bix, y) be the statement “y is the best friend of r." To translate the sen¬ 
tence in the example, note that it says that for every person x there is another person y 
such that y is the best friend of .r and that if ; is a person other than y, then z is not the 
best friend of x. Consequently, we can translate the sentence as 

V.v3y V: (B(x, y) A((^ y) -• B(x, z))). ■ 

EXAMPLE 17 

Express the statement “If somebody is female and is a parent, then this person is some¬ 
one’s mother" as a logical expression, 

Solution: Let F(x) be the statement “r is female,” let P(x) be the statement u x is a 
parent," and let M(x, y) be the statement “.v is the mother of y.” Since the statement in 
the example pertains to all people, we can write it symbolically as 

Vr (i ‘F(x) A /’it)) — 3y M( jt, y)), ■ 

EXAMPLE 18 

Use quantifiers to express the statement “There is a woman who has taken a flight on 
every airline in the world." 

Solution: Ixt P(w. f) be “w has taken/ 1 ’ and Q(J, a ) be “f is a flight on ax We can 
express the statement as 

3wVa3f{P(w,f)AQ(f,a)), 

where the universes of discourse for w, /, and a consist of all the women in the world, 
all airplane flights, and all airlines, respectively. 
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The statement could also be expressed as 
3wVa3 / R(w, f,a ), 

where R(w,f, a) is “w has taken / on a Although this is more compact, it some¬ 
what obscures the relationships between the variables. Consequently, the first solution 
is usually preferable. ■ 


As mentioned earlier, quantifiers are often used in the definition of mathematical 
concepts. One example that you may be familiar with is the concept of limit, which is 
important in calculus. 


EXAMPLE 19 (Calculus required) Express the definition of a limit using quantifiers. 

Solution; Recall that the definition of tire statement 
lim/W = L 

is: For every real number e > 0 there exists a real number 8 > 0 such that |/(.r) - 
L\ < e whenever 0 < |x - a\ < 5. This definition of a limit can be phrased in terms of 
quantifiers by 

Ve 3-5 V.\(0 < j.t ■ <t\ < 8 |/(.v) - /-I < e). 

where the universe of discourse for the variables 8 and e is the set of positive real 
numbers and for v is the set of real numbers. 

This definition can also be expressed as 

Ve > 035 > 0Vx(0 < |jt - a\ < 5 -* \f{x) - L\ < e> 

when the universe of discourse for the variables e and 8 is the set of all real numbers, 
rather than the set of positive real numbers. ■ 

EXAMPLES FROM LEWIS CARROLL (optional) 


Lewis Carroll (really C. L, Dodgson w riting under a pseudonym), the author of Alice in 
Wondedarui, is also the author of several works on symbolic logic. His books contain 
many examples of reasoning using quantifiers. The next two examples come from his 
book Symbolic Logic; other examples from that book are given in the exercise set at 
the end of this section. These examples illustrate how- quantifiers are used to express 
various types of statements. 


Charles Lutwidge Dodgson II832-1898), We kaow Charles Dodgson as Lewis Carroll —the pseudonym 
W{ib he uscd m his writings on logic. Dodgson, the son of a clergyman, was the third of 11 children, all of whom 
stuttered. He was uncomfortable in the company of rid nils and is said to have spoken without stuttering nriffT 
to young girls, many of whom he entertained, corresponded with, and photographed (often in the nudei. 
Although attracted to young girls, he was extremely puritanical and religious. His friendship with the three 
young daughters uf Dean Liddell led to his writing Alice in Wonderland, which brought him money anti 
tame. 

Dodgson graduated from Oxford in 1854 and obtained his master of arts degree in 1857. He was ap¬ 
pointed lecturer in mathematics at Christ Church College. Oxford, in 1855. He was ordained in the Church 
of England in 1861 but never practiced his ministry. His writings include articles and books on geome¬ 
try. determinants, and the mathematics of tournaments and elections. (He also used the pseudonym Lewis 
Carroll for his many works on recreational logic ; 


PDF created with pdfFactory trial version www.pdffactorv.com 





1,3 Predicates and Quantifiers 29 


EXAMPLE 2(1 

Consider the following statements. The first two are called premises and the third is 
called the conclusion. The entire set is called an argument. 

“All lions axe fierce.” 

“Some lions do nol drink coffee.” 

“Some fierce creatures do not drink coffee.” 

(In Section 3.1 we will discuss the issue of determining whether the conclusion is a 
valid consequence of the premises. In this example, it is.) Let R\x). Q{x), and R(x) be 
the statements A is a lion,” A is fierce,” and A drinks coffee,” respectively. Assuming 
that the universe of discourse is the set of all creatures, express the statements in the 
argument using quantifiers and P(x), Q(x), and R(x). 

Solution: We can express these statements as: 

V.v(Pfv) - thvi: 

3 v. Pi r i A R(x)}. 

H i: (J;.v) A R(x)i 

Notice that the second statement cannot be written as 3 si A n —»■ ~i R(x\). The reason 
is that Fix) -i R(x) is true whenever x is not a lion, so that 3_r(P(jr) —*■ i/f(,c))is 

true as long as there is at least one creature that is not a lion, even if every lion drinks 
coffee. Similarly, the third statement cannot be written as 

3,r(£U) - ->R(x)\. U 

EXAMPLE 21 

Consider the following statements, of which the first three are premises and the fourth 
is a valid conclusion. 

“Alt hummingbirds are richly colored.” 

“No large birds live on honey.” 

“Birds that do not live on honey arc dull in color.” 

“Hummingbirds are small.” 

Let P(x), <JiR(x), and S(x) be the statements A is a hummingbird,” A is large,” 
A lives on honey,” and A is richly colored,” respectively. Assuming that the universe of 
discourse is the set of all birds, express the statements in the argument using quantifiers 
and P(x}, Q(x), R(x), and S(.rl, 

Solution: We can express the statements in the argument as: 

Vx(P(x) — 5(A). 

Tv! (9; A A A’! vii 

Vx(-iR(x) -iS(.t)). 

VxiP(x) - -ngfjQ). 

(Note we have assumed that ‘‘small” is the same as “not large” and that “dull in color” is 
the same as “not richly colored.” To show that the fourth statement is a valid conclusion 
of the first three, we need to use rules of inference that wilt be discussed in Section 3.1.) 

■ 


BINDING VARIABLES 

When a quantifier is used on the variable t or when we assign a value to this variable, we 
say that this occurrence of the variable is bound. An occurrence of a variable that is not 
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EXAMPLE 22 


EXAMPLE 23 


bound by a quantifier or set equal to a particular value is said to be free. All the variables 
that occur in a propositional function must he bound to turn it into a proposition. This 
can be done using a combination of universal quantifiers, existential quantifiers, and 
value assignments. 

Many mathematical statements involve multiple quantifications of propositional 
functions involving more than one variable. It is important to note that the order of the 
quantifiers is important, unless all the quantifiers are universal quantifiers or all are 
existential quantifiers. These remarks are illustrated by Examples 22, 23, and 24. In 
each of these examples the universe of discourse for each variable is the set of real 
numbers. 


Let Pi. r, v) be the statement “r + v = v+x” What i.s the truth value of the quantification 
VtVv/'icv) 1 

Solution: The quantification 
VrVyPU, v) 
denotes the proposition 

“For all real numbers x and for all real numbers y, it is tme that x + y - y + x." 
Since P(x. v) is true for all real numbers v and v, the proposition V.rVy P(x, y) is true. 

a 


Let Q(x, y) denote "x + y = 0.” What arc the truth values of the quantifications 
3y Vo Q(x. y) and V.v 3y Q(x. y)'} 

Solution: Tire quantification 
3y V,v Q(jc. v) 

denotes the proposition • 

“There is a real number y such that for every real number x, Q(x, y) is true.” 

No matter what value of y is chosen, there is only one value of x for winch x + v = 0. 
Since there i.s no real numbery such that x + y - 0 for all real numbers.':, the statement 
3yV.tQ(.v, v) is false. 

The quantification 

V.v 3 ; Q(x, y) 

denotes the proposition 

“For every real number x there is a real number y such that Qix, v) is true," 

Given a real number x, there is a real number y such that x + y - 0; namely, y - -x. 
Hence, the statement Vjc 3y Q(x, y) is true. ■ 


Example 23 illustrates that the order in which quantifiers appear makes a differ¬ 
ence. The statement 3y V.v P(x, y) and Vx 3y P(x. y) are not logically equivalent. The 
statement 3y V.cFU, y) is true if and only if there is a v that makes T(.r, y) true for 
every x. So, for this statement to be true, there must be a particular value of y for 
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which Pi t, y) is true regardless of the choice of ,r. On the other hand, Vx 3y P(x, y) is 
true if and only if for every value of x there is a value of y for which P(x, v'l is true. So, 
for this statement to be true, no matter which x you choose, there must be a value of y 
(possibly depending on the x you choose) for which P(x, y) is true. In other words, in 
the second case y can depend on .t, whereas in the first case y is a constant independent 
of x. 

From these observations, it follows that if 3 y Vx P(.r, y) is true, then Vx 3y Fix, y) 
must also be true. However, if Va 3y P(x. y) is true, it is not necessary for 3y Vx Pi t, y) 
to be true. (See Supplementary Exercises 8 and 10 at the end of this chapter.) 

In working with quantifications of more than one variable, it is sometimes helpful 
to think in terms of nested loops. (Of course, if there are infinitely many elements in 
the universe of discourse of some variable, we cannot actually loop through all values. 
Nevertheless, this way of thinking is helpful in understanding nested quantifiers,) For 
example, to see whether VxVy P(x, y) is true, we loop through the values forx, and for 
each x we loop through the values for y. If we find that P(x, y ) is true for all values for x 
and y, we have determined that Vx Vy P(x, y) is true. If we ever hit a value x for which 
we hit a value y for which P(x, y) is false, we have shown that Vx Vy P(x, y) is false. 

Similarly, to determine whether Vx 3y P(x, y) is true, we loop through the values 
for x. For each x we loop through the values for y until we find a y for which P(x, y) is 
true. If for all x we hit such a y, then Vx3y P(x, v) is true; if for some x we never hit 
such a y, then Vx 3y Pix, y) is false. 

To see whether 3x Vy P(x, y) is true, we loop through the values for x until we find 
an x for which P(x, y) is always tme when wedoop through all values for y. Once we 
find such an x, we know' that 3xVy P(x, y) is true. If we never hit such an x, then we 
know that 3x Vy P(x, y) is false. 

Finally, to see whether 3x3y P(x, y) is true, we loop through the values for x, 
where for each x we loop through the values for y until we hit an x for which we hit a v 
for which Pix. y) is tme. The statement 3x 3> P(x, y) is false only if we never hit an x 
for which we hit ay such that P(x, y) is true. 

Table 2 summarizes the meanings of the different possible quantifications involving 
two variables. 

Quantifications of more than two variables are also common, as Example 24 illus¬ 
trates. 


TABLE 2 Quantifications of'Two Variables. 

Statement When True? 

When False? 

Vx Vy Pi x y) 

V v Vx P( x, y) 

Pix. y) is true for every pair 

x, y. 

There is a pair x, v for 
which Pix, y) is false. 

Vx 3 v Pix, v) 

For every x there is a y lor 
which Pi x, y) is true. 

There is an x such that 

P(x, yi is false for every y. 

IxVvPU v) 

There is anx lor which Pi x. y) 
is tme for every y, 

For every x there is a y for 
which Pix, y) is false. 

3x3yP{x,y) 

3v3x Put >) 

There is a pair r, y for which 
P[x, y) is true. 

Pix, y) is false for every 
pair x, y. 
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EXAMPLE 24 


Let Q(x, v, j) be the statement “i + y - z" What are the truth values of the statements 
Vx Vv 3- Q(x, y, ;) and HcVrVv Q(x, y, Z )1 

Solution: Suppose that a and y are assigned values. Then, there exists a real number j 
such that x + y = z. Consequently, the quantification 

VjV.v3 zQ(x,y, z), 

which is the statement 

“For all real numbers x and for all real numbers v there is a real number ; such 
that x + y = 

is true. The order of the quantification here is important, since the quantification 
3;V.\: Vvgf.v, y. z). 
which is the statement 

“There is a real number z such that for all real numbers x and for all real numbers 
v it is true that x + y = z.” 

is false, since there is no value of; that satisfies the equation x + v = z fur all values 
of x and y. ■ 

NEGATIONS 


We will often want to consider the negation of a quantified expression. For instance, 
consider the negation of the statement 

"Every student in the class has taken a course in calculus." 

This statement is a universal quantification, namely. 

V.v P{x), 

where P(.x) is the statement “x has taken a course in calculus." The negation of this 
statement is “It is no! the ease that every 1 student in the class has taken a course in 
calculus." This is equivalent to 'There is a student in the class who has not taken a 
course in calculus." And this is simply the existential quantification of the negation of 
the original propositional function, namely, 

3.v <P(xl 

This example illustrates the following equivalence: 

-oMxPix) <=> 

Suppose we wish to negate an existential quantification. For instance, consider the 
proposition “There is a student in this class who has taken a course in calculus." This 
is the existential quantification 

d .'W i 

where Qix) is the statement “x has taken a course in calculus.” The negation of this 
statement is the proposition "It is not the ease that there is a student in this class who 
has taken a course in calculus” This is equivalent to “Every student in this class has 
not taken calculus,” which is just the universal quantification of the negation of the 
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Negation 

Equivalent Statement 

When h Negation True? 

When False ? 

•Hi Pix) 

< 

* 

J 

y 

Pix) is false for every x. 

There is an x for which 
Pix) is true. 

-Vx Ax) 

Ht- P(x) 

There is an x for which 

Pix) is false. 

P(x) is true for every x. 


original propositional function, or, phrased in the language of quantifiers, 
Vjf -nQ( x). 

This example illustrates the equivalence 
-3xQ(x) <=> Vjt -iQ(xi 
Negations of quantifiers are summarized in Table 3. 


Exercises 

1. Let Fi x) denote the statement "a < 4," What are the 
truth values of the following? 

a) P(Q) b) P( 4} ei P{ 6) 

2. Let Pi, x) be the statement “the word x eon tains the letter 
uF What are the truth values of the following? 

a) /’(orange) b) /'(lemon) 
e) /'(true) d) P( false) 

3. Let Qix, v) denote the statement “,v is the capital of y." 
What are the truth values of the following? 

a) (/(Denver, Colorado) 

b) (/(Detroit. Michigan) 

c) (/(Massachusetts, Boston) 

d) (/(New York, New York) 

4. State the value of x after the statement if P{x) then 
a : ~ 1 is executed, where P{. r) is the statement “i > 
L" if the value of x when this statement is reached is 
a) x = 0 b) a • 1 c) a = 2 

5. Lei Fix) be the statement “x spends more than five 
hours every weekday in class," where the universe of 
discourse for x is the set of students. Express each of 
the following quantifications in English. 

a) 3xP(x) bj Vx :P(x) 

c) 3x-> Fix) d) Vx ->P(x) 

6. Let Fix, y) be the statement “x has taken class v.” 
where the universe of discourse for x is the set of all 
students in your class and for y is the set of all com¬ 
puter science courses ai your school. Express each of 
the following quantifications in English, 

at 3x 3y P(x, v) b) BxVyPix, y) 

c) Vx 3y P(x, y) d) 3vVjcPUv) 

e) Vy 3x Fix. y) f) Vx Vy P(x, y) 


7. Let IVU, y) mean that x has visited y, where the uni¬ 
verse of discourse for x is the set of all students in your 
school and the universe of discourse for y is the set of 
all Web sites. Express each of the following statements 
by a simple English sentence. 

a) W(Sarah Smith, www.att.com) 

b) 3x W(x, www'.imdb.org) 
e) 3y WfJose Orez, y) 

d) 3y (WfAshok Puri, y)A BY Cindy Yoon, y)) 

e) 3vtf-(y ¥=■ (David Belcher)A(W'fDavid Belcher, z) 

- K'f>. :).J 

f) 3x3y Vz((x t* y) A (W(x, ,) * • H’( v. 

8. Let C( x, y) mean that x is enrolled in y, where the uni¬ 
verse of discourse for x is the set of all students in your 
school and the universe of discourse for y is the set of 
all classes being given at your school. Express each of 
the following statements by a simple English sentence 

a) a Randy Goldberg, CS 252) 

b) 3xC(.v. Math 095) 
e) 3 y CiCarol Sitea, y) 

d) 3x(C(,r, Math 222 sA C(x,«S 252)) 

e) 3.T 3y V:(i.t * y) A iCu.:) C{y, z)'» 

f) 3x 3y Vz((x # y) A (C(x. z) +* C(y, z))) 

9. Let P{x) be the statement “x can speak Russian" and 
let Q(x) be the statement "x knows the computer lan¬ 
guage C++,” Express each of the foliowing sentences 
in terms of P(x), Qix), quantifiers, mid logical connec¬ 
tives. For the universe of discourse for quantifiers use 
the set of all students at your school. 

a) There is a student at your school who can speak 
Russian and who knows C++. 
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hi There is a student at your school who can speak 
Russian but who doesn’t know C++, 

c) tv cry student at your school either can speak Rus¬ 
sian or knows C + +, 

d> No student at your school can speak Russian or 
knows C • •. 

10. Let Qix. y) be the statement *tr has been a contestant on 
v." Express each of the following sentences in terms of 
Qt x, y), quantifiers, and logical connectives, where the 
universe of discourse for x is the set of all students at 
your school and lor v is the set of all quiz show's on 
television, 

a I There is a student al your school who has been a 
contestant on a television quiz show, 
hi No student at your school lias ever been a contestant 
on a television quiz show. 

c) Th.etc is a student at your school who has been a 
contestant on Jeopardy and on Wheelsf Fortune. 

d) Every television quiz show has had a student from 
your school as a contestant, 

e) At least two students from your school have been 
contestants on Jeapardv. 

11. Let Hx, y ) he the statement "x loves y,” where the uni¬ 
verse of discourse fuT both r and v is the set of all peo¬ 
ple in the world. Use quantifiers to express each of the 
following statements: 

a) Everybody loves Jerry. 

h) Everybody loves somebody. 

e) There is somebody whom every body loves, 

d) Nobody loves everybody, 

e) There is somebody whom Lydia does not love. 

f) There is somebody whom no one loves, 

g) There is exactly one person whom everybody loves, 

h) There are exactly two people whom Lynn loves, 
i i Everyone loves himself or herself. 

j) There is someone who loves no one besides himself 
or herself. 

12. Let F( v, v) be the statement “x can fool y." where the 
universe of discourse is the set of all people in the 
world. Use quantifiers to express each of the following 
statements: 

a) Everybody can fool bred, 
h) Evelyn can fool everybody. 

c) Everybody can fool somebody. 

d) There ts no one who can fool everybody, 
el ’Everyone can be fooled by somebody. 

f) No one can tool both Fred and Jerry. 

g) Nancy can fool exactly two people. 

h) There is exactly one person whom everybody can 
fool. 

i) No ceie can fool himself or herself. 

j) There is someone who can fool exactly one person 
besides himself or herself, 

13. Let i’u) he the predicate “x is a student," Full the 
predicate ",i is a faculty member," and Af.t.y) the pred¬ 
icate *‘.v has asked y a question,” where the universe of 


discourse is the set of all people associated with your 
school. Use quantifiers to express each of the following 
statements. 

a,l Lois has asked Professor Michaels a question, 
bj Every student hits asked Professor Gross a question. 

c) Every faculty member has either asked Professor 
Miller a question or been asked a question by Pro¬ 
fessor Miller. 

d) Some student has not asked any faculty member a 
question, 

e) There is a faculty member who has never been 
asked a question by a student 

f) Some student has asked every faculty member a 
question. 

g) There is a faculty member who has asked every 
other faculty member a question. 

h) Some student has never been asked a question by a 
facility, member. 

14, Let l(x) be the statement " t has an Internet connec¬ 
tion'’ and Cpr.y) be- the statement “x and v have chat¬ 
ted over the Internet," where the universe of discourse 
forthe variables x and v is the set of all students in your 
class. Use quantifiers to express each of Ihe following 
statements. 

a) Jerry does not have an Internet connection, 
h) Rachel has not Chatted over the Internet with 
Chelsea. 

c) Jan and Sharon have never chatted over the Inter¬ 
net. 

d) No one in the class has chatted with Bob 

e) Sanjay has chatted with everyone except Joseph. 
f> Someone in your class dues not have an Interne! 

connection. 

g) Not everyone in your class has an Internet connec¬ 
tion. 

h) Exactly one student in your class has an Internet 
connection, 

i) Everyone except one student in your class has an 
internet connection, 

j) Everyone in your class with an Internet connection 
has chatted over the Internet with at least one other 
student in your class. 

k) Someone in your class has an Internet connection 
but has not chatted with anyone else in your class. 

l) There are two students in your class who have not 
chatted with over the Internet, 

m) There is a student in your class who has chatted with 
everyone in your class over the Internet, 
ti) There are at least two students in your class who 
have not charted with the same person in your class, 
u) There are two students in the class who between 
them have chatted with everyone else in the class. 

IS. Let Mix, y) be "< has sent v an e-mail message” and 
T( t, y) he “.v has telephoned v,” where the universe of 
discourse is the set of all students in your class. Use 
quantifiers to express each of the following statements. 
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(Assume that all e-mail messages lhat were sent are 
received, which is rot the way things often work-1 

a) Chon has never sent an e-mail message to Koko. 
bi Arlene has never sent an e-mail message In or tele¬ 
phoned Sarah. 

r) Jose has never received an e-mail message from 
Deborah. 

d) Every student in your class has sent e-mail message 
to Ken 

e) No one in your class has telephoned Nina. 

f) Everyone in class has either telephoned Avi or sent 
him an e-mail message, 

g> There is a student in your class who has sent every¬ 
one else in your class an e-mail message, 
h) There is someone in your class who has either sent 
an e-mail message or telephoned everyone else in 
your class, 

h There are two students in your class who have sent 
each other e-mail messages. 

j) There is a student who has sent himself or- herself 
an e-mail message. 

k) There is a student in your class who has not received 
an e-mail message from anyone else in the class anti 
who has not been called by any other student in the 
class. 

11 Every student in the class has either received an e- 
mtul message or received a telephone call from an¬ 
other student in the class. 

m) There- are at least two students in yam class such 
that one student has sent the other t'-mail and the 
second student has telephoned the first student. 

n) There are two students in your class who between 
them have sent an e-mail message to or telephoned 
everyone else in the dass. 

16. U sc quantifiers to express each of the following state¬ 
ments. 

a) There is a student in this class who can speak Hi ndi. 

b) Every student m this class knows how to drive a car, 

e) Some student in this class has visited Alaska but 

has not visited Hawaii. 

d) All students it) this class have learned at least cue 
programming language. 

e) There is a student in this class who has taken ev¬ 
ery course offered by one of the departments in this 
school. 

f) Stunt student in this class grew up in the same town 
as exactly one other student in this class. 

g) Eicry student in this class has chatted with at least 
one other student in at least one on-line chat group. 

17. Use quantifiers to express the following statements, 
aj Every computer science student needs a course in 

discrete mathematics. 

hi There is a student in this class who owns a personal 
computer. 

t) Every student in this class has taken at least one 
computer science course. 


d) There ts a student in this class who has taken at least 
one course in computer science, 
ef Every student in this class has been in every build¬ 
ing on campus, 

f! There is a student in this class who has been in ev¬ 
ery room of at least one building on campus, 

g) Every student in this class has been in at least one 
room of every building on campus. 

IS. A discrete mathematics class contains 1 mathematics 
major who is a freshman, 12 mathematics majors who 
are sophomores. 15 computer science majors who are 
sop horn ores, 2 mathematics majors who are juniors, 2 
computer science majors who are juniors, and 1 com¬ 
puter science major w ho is a senior. Express each of the 
fallow ing statements in terms of quantifiers and then 
determine its truth value, 
a) There is a student in the class who is a junior, 
b'l Every student m the class is a computer science ma¬ 
jor. 

cf There is a student in the class who is neither a math¬ 
ematics major nor a junior. 

d) Every' student in the class is either a sophomore or 
a computer science major, 

e) There is a major such that there is a student in the 
class in every year of study with that major. 

19. Let Pi x) be the statement “a = .\ 2 ” If the universe of 
discourse is the set of integers, what arc the truth values 
of the following? 

a) T(0) hi P{ Ij e) /’:?• 

d) Pi - !) ei 3x P(x) f| Vi P{x) 

20. Let gu. yi be the statement "x -t y = x - y." If the 
universe of discourse for both variables is the set of in¬ 
tegers, what are the truth values of the following? 

a) «U> b) 6(2,0) 

c) Vv£)(l. v) d) 3 a Q(.x. 2) 

e) 3.13 v 0t-t, y) f) V.v 3v Qij, v) 

g) 3> V t <>( i. h) Vy-3x0(,i,y) 

i) V.r Vy (J(x, yj 

21. Determine the truth value of each of the following 
statements if the universe of discourse for all variables 
is the set of all integers. 

a) Vttfjr i 0) bi 3n(n- - 2) 

c) Vn(.*r i n) d) Vw 3m(tr < mi 

e) 3uVm(« c jw a > f) V«3m(» t- m = l)> 

r! 3nVm(m« --- mi h) 3n3ni[rr + m 2 = 5) 

ij 3n 3m(n : -f nr - 6) 

ji 3 h 3m(it + m "■ 4 A n - m = I) 

k) 3n 3 m(n + m - 4 A n - m - 2) 

1.1 V n V m 3 p(p - (m + n}/2) 

22. Determine the truth value of each of the following 
statements if the universe of discourse of each variable 
is the set of real numbers, 

a) 3.,. t ; = 2) 


e) Vx 3y(?r v) 

01 3 _vVyf.iv - (fl 
«i V ■ * 1)3vi i, ^ ]) 


b) 3x i- = -I) 
d) V.r 3 vf.r - r) 

0 3.r3.v(.r + y A y -t v) 
hi 3xVv^0(rv ■= 1) 
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i) V.v3via t v - 1) 

j) 3x3vU + 2v - 2A2 a' + 4v = 5) 

k) V.» 3>tt t >• = 2A2x - v = I) 

l) Va Vv 3:t: = U - vj/2) 

23. Suppose the universe of discourse of the proposi¬ 
tional function P(x, v) consists of pairs a and y, where 
v is 1,2, or 3 and v is 1. 2, or 3. Write out the 
following propositions using disjunctions and con¬ 
junctions. 

a) 23 c /*• ; 1) hi Vv 1 y) 

c) ViV r .1 d) j.j,/ i. y) 

e> 3 x V'r P{ a. y) 0 V;y 3 x Pi X, y ) 

24. Rewrite each of the following statements so that nega¬ 
tions appear only within predicates (that is. so that no 
negation is outside a quantifier or an express to n involv¬ 
ing logical connectives}. 

a) ! 3v 3x/ > (.v,y) 

b) -iV.vByPU-y) 

c) ~ Q‘ v) A V t At 

d) -i 3y (3 x R{ x, y) y Vx S(x. y)) 

«l “ 3y 1 V.i 3: T: x, y,:) y 3,t Vs Ulx, y,4) 

25. Rewrite each of the following statements so that nega¬ 
tions- appear only within predicates (that is, so that no 
negation is outside a quantifier or an expression involv¬ 
ing logical connectives). 

a) iVxVyAU.v) 
h) -iVv3a PU.y) 

e) - Vy V.v i Pi r, y) y (?(,(,>')} 

dl -U3.t3v V.vVv{h.v,y;.| 

el -iVriJyV: Plx.y.zl A3;Vy P(x,y,z)) 

2b. Express each of die following statements using quan¬ 
tifiers. Then form the negation of the statement so that 
no negation is to the left of a quantilier. Next, express 
the negation in simple English. (Do not simply use the 
words “It is not the case that.") 
a) All dogs have fleas. 

hi No one has lost more than one thousand dollars 
playing the lottery. 

e) There is u student in this class who has chatted with 
exactly one other student. 

dl No student in this class has sent e-mail to exactly 
two other students m this class. 

*) Some student has solved every exercise in this 
book. 

fl No student has solved at least on e exercise in every 
section of this book. 

27, Express each of the following statements using quanti¬ 
fiers. Then form the negation of the statement, so that 
no negation is to the left of a quantifier. Next, express 
the negation in simple English. (Do not simply use the 
words “It is not the case that.’’) 
a) There is no dog that can talk, 
bl There is no one in this class who knows french and 
Russian. 

c) Every student in this class has taken exactly two 
mathematics classes at this school. 


d) Someone has visited every country in the world 
except Libya. 

el No one has climbed every mountain in the Hi¬ 
malayas. 

f) Every movie actor has either been in a movie with 
Kevin Bacon or has been in a movie with someone 
who has been in a movie with Kevin Bacon. 

2& Express the negations of the following propositions us¬ 
ing quantifiers, and in English, 
a) Every student in this class likes mathematics, 
h) There is a student in this class who has never seen 
a computer. 

c) There is a student in this class who has taken every 
mathematics course offered at this school. 

d) There is a student in this class who has been in at 
least one room of every building on campus. 

29. Use quantifiers to express the associative law for mul¬ 
tiplication of real numbers. 

30. Use quanti fiers to express the distributive laws of mul¬ 
tiplication over addition for real numbers. 

Exercises 31-34 arc based on questions found in the book 
Symbolic Logic by Lewis Carroll. 

31. Let Pi ul. (Ax), and Atx) be the statements "_r is a pro¬ 
fessor," “x is ignorant,and “a is vain," respectively. 
Express each of the following statements using quan¬ 
tifiers; logical connectives; and P{x). Q{:< t, and Rix). 
where the universe of discourse is the set of all people. 

a) No prolessors are ignorant. 

b) All ignorant people are vain. 

e) No professors arc vain. 

d) Does (c) follow from (a) and (b)? If not. is there a 
correct conclusion7 

32. Let I'(x). Q(x), and R(x) be the statements “x is a clear 
explanation," “x is satisfactory 1 ,” and "x is an excuse," 
respectively. Suppose that the universe of discourse for 
x is the set of all English text, Express each of the 
following statements using quantifiers; logical connec¬ 
tives; and P(x), Q{x), and ff(jr), 

a) All clear explanations are satisfactory. 

b) Some excuses are unsatisfactory. 

cl Some excuses are nol clear explanations. 

*d) Does fcj follow from (a) and (EV If not, is there a 
correct conclusion? 

33. I.ct A(y), £)(,rI, A(,r), and.5 ’(a) be the,statemenls 'T isa 
baby," “x is logical," “a is able to manage a crocodile," 
and "x is despised,” respectively. Suppose that the uni¬ 
verse of discourse is the set of all people. Express each 
of the following statements using quiintifiers; logical 
connectives; and /’lx), G(x), R(x). and Six). 

a) Babies are illogical. 

bl Nobody is despised who can manage a crocodile. 

c) Illogical persons are despised. 

d) Babies cannot manage crocodiles. 

*e) Does (d) follow from (a), lb), and (c)? If not, is there 
a correct conclusion? 
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34. 1 art Fix), (?(,r), R(x), and .S'fjc) be the statements “i 
is a duck," “x is one of my poultry," "x is an officer," 
and "r is willing to waltz.” respectively. Express each 
of the following statements using quantifiers: logical 
connectives; and P(x). Q{x), R(x), and Six). 

a) No ducks are willing to waltz. 

b) No officers ever decline to waltz, 

c) All my poultry are ducks. 

d) My poultry are not officers. 

*e) Does (d) follow from (a), ib), and (e)? If not, is there 
a correct conclusion? 

35. Show that the statements -i3xVyF(x, y) and 
V.r 3 e -i F(.i, y) have the same truth value. 

34. Show that V.r (P(x) A Q(x)) and Vx P(x) A VxQ(x) 
have the same truth value 

37. Show that 3x(F(.v) V' Qi*)} arid 3xF(x)y 3 t Q{ a • 
have the same truth value. 

38. Establish the following logical equivalences, where .4 
is a proposition not involving any quanti Tiers, 

a) (V.i P(x))yA VxiPix) y A) 
hi i Jx /Vv;A. A <A=> 3.t(f(.t)yA) 

39. Establish the following logical equivalences, where .4 
is a proposition not involving any quantifiers. 

a) (V.t F(.v)) A A <-> \/x(P[x)/\A) 
hi (3.v F(xl] A A v > 3x(/ J (x) A .4) 

40. Show that Vx P(xlyVx Q(x) and V.r [ P{x ) y Q[x )) are 
not logically equivalent. 

41. Show that 3 x P(x) A 3i i.i. t i and A.xtP: xi ‘ Q \)) are 
not logically equivalent, 

*42. Show that Va F:.v: . V.x Q[x) and V.r V v (Pi a 1 y (Ay)I 
arc logic ally equivalent. (The new variable y is used to 
combine the quantifications correctly,) 

*43. a) Show that Vx Put A 3 a (h x < and Vx 3y (Pi x) A 
(J> . i i rat- equivalent. 

h) Show that VxPfxjy 3.x Q{x) and V.r3y (Fix) V 
Qiy)) are equivalent. 

44. The rotation 3!x Fix) denotes the proposition 

’There exists a unique x such that P{x) is true." 

If the universe of discourse is the set of integers, what 
are the truth values of the fallowing? 
a) T >i • > 1) 

M 3!.vU’ = 1) 

c) 3!x(x -l- 3 ■= 2x) 

d) 3!x(x - x + 1) 

45. What arc the truth values of the following statements? 
a) 3!x Fix) — 3 xP(x) 

lx V.r Fix) -<■ 3!x F(.r) 
c) 31.r -iF(.r) — -iVxPt.r) 

46. Write out the quantification 3Ltf(.v). where the uni¬ 
verse of discourse consists of the integers 1,2, and 3, 
in terms of negations, conjunctions, and disjunctions. 

*47. Express the quantification 3 A P( r) using universal 
quantifications, existential quantifications, and logical 
operators. 


A statement is in pronex normal form (PNF) if and only 
if it is of the form 

Q]X } Qx\ 2 "-Qt-nPU\. xi .xi). 

where each Q„ i - 1,2is cither the existential 

quantifier or the universal quantifier, and F(X|.,yi 

is a predicate involving no quantifiers. For example. 
3.v VvlFu, y) A jgfy?) is in prenex normal form, whereas 
j x P(x) y Vx Qfx} is not (since the quanti tiers do not all 
occur first). 

Every statement formed from propositional variables, 
predicates. T, and F using logical connectives and quan¬ 
tifiers is equivalent to a statement in prenex normal Form. 
Exercise 49 asks for a proof of this fact. 

*48. Pul the following statements in prenex normal form. 
[Him: Use logical equivalence from Tables 5 and 6 in 
Section 1.2, Table 2 in this section, and Exercises 36- 
39 and 42 >43 in this section.) 

a) 3x P(x) V 3 1 Q(x) y A, where A is a proposition 
not involving any quantifiers. 

b) -• V x An , Vx (M'. 

c) 3.x F. x i — 3x Q[x i 

**49. Show how to transform an arbitrary statement to a state¬ 
ment in prenex normal form that is equivalent to: the 
given statement. 

A real number x is called an upper bound of a set S of real 
numbers if x is greater than or equal to every member of S. 
The real number x is called the least upper bound of a set 
S of real numbers if x is an upper bound of S and x is less 
than or equal to every upper bound of S'; if the least upper 
bound of a seL S exists, it is unique. 

50. a) Using quantifiers, express the fact that x is an upper 

bound of S. 

b) Using quantifiers, express the fact that x is the least 
upper bound of S. 

51. (Calculus? required) Using quantifiers, express the fact 
that liny - fix) does not exist. 

The statement - L means that for every posi¬ 

tive real number e there is a positive integer N such that 
|a„ — L\ < e whenever w > JV. 

52. (Calculus required) Use quanti fiers to express the state¬ 
ment that !im„_, «„ - L. 

53. (Calculus required) Use quantifiers to express the state¬ 
ment that !im„ a„ does not exist. 

54. (Calculus required) Use quantifiers to express the fol¬ 
lowing definition: A sequence {ci„} is a Cauchy se¬ 
quence if for every real number € > 0 there exists a 
positive integer N such (hat |« m - «„}<€ for every' pair 
of positive integers m and n with m > N and n > .V. 

55. (Calculus required) Use quantifiers and logical connec¬ 
tives to express the following definition A number L is 
the limit superior of a sequence (a,,} if for every rcai 
number e > 0, a„ > L - <s for infinitely many n and 
a* > L e for only finitely many n. 
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L4 

Sets 


INTRODUCTION 

We will study a wide variety of discrete structures in this book. These include rela¬ 
tions, which consist of ordered pairs of elements: combinations, which are unordered 
collections of elements; and graphs, which are sets of vertices and edges connecting 
vertices. Moreover, we will illustrate how these and other discrete structures are used 
in modeling and problem solving. In particular', many examples of the use of discrete 
structures in the storage, communication, and manipulation of data wail be described. In 
this section wc study the fundamental discrete structure upon which all other discrete 
structures are built, namely, the set. 

Sets are used to group objects together. Often, the objects in a set have similar 
properties. For instance, all the students who are currently enrolled in your school make 
up a set. Likewise, all the students currently taking a course in discrete mathematics at 
any school make up a set. In addition, those students enrolled in your school who are 
taking a course in discrete mathematics form a set that can be obtained by taking the 
elements common to the first two collections. The language of sets is a means to study 
such collections in an organized fashion. 

Note that the term object has been used without specifying what an object is. This 
web description of a set as a collection of objects, based on the intuitive notion of an object, 
was first stated by the German mathematician Georg Cantor in 1S95. The theory that 
results from this intuitive definition of a set leads to paradoxes, or logical inconsisten¬ 
cies, as the English philosopher Bertrand Russell showed in 1902 (see Exercise 26 for a 
description of one of these paradoxes). These logical inconsistencies can be avoided by 
building set theory' starting with basic assumptions, called axioms. We will use Can¬ 
tor’s original version of set theory, known as naive,set theory, without developing an 
axiomatic version of set theory', since all sets considered in this book can be treated 
consistently using Cantor’s original theory. 

We now proceed with our discussion of sets. 

DEFINITION 1. The objects in a set are also called (he elements, m mmbem 'of 
the set. A set is said to contain its elements. 


(Jeorg Cantor (1X45-19 IN). Georg Can lor unw born in St. Petersburg. Russia, where his fattier was a 
W€b successful merchant. Cantor developed his interest in mathematics in his teens. He began his university 
Studies in Zurich irt 1X62, but when his father died he left Zurich. He continued his university studies at the 
University fit Berlin in 1863, where he studied under the eminent mathematicians Weiers trass. Rummer, 
and Kronerker He received his doctor's degree in 1867 after having written a dissertation on number theory. 
C antor assumed a position at the University of Halle in 1869, where he continued working until hit, death. 

Cantor is considered the founder of set theory. His contributions in this area include the discovery that 
the set oi real numbers is uncountable. He is also noted for his many important contributions to analysis. 
Cantor also was interested in philosophy and wrote papers relating ho theory of sets with metaphysics. 

Cantor married in 1874 and had five children. His melancholy temperament was balanced by his 
wile's happy disposition. Although he received a large inheritance from his father, he was pooHv paid as 
a professor. To mitigate this, he tried to obtain a heller-paying position at the University of Berlin. His 
appointment there was blocked by Kroncckcr, who did not agree with Cantor’s views on set theory Cantor 
suffered from menial illness throughout the later years of his life. He died in 1918 in a psychiatric clinic. 
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There lire several ways to describe a set. One way is to list all the members of a set, 
when this is possible. We use a notation where all members of the set are listed between 
braces. For example, the notation {a, b, c,d) represents the set with the four elements 
a, b. c, and cl 

EXAMPLE 1 

The set V of all vowels in the English alphabet can written as V = {a, e, i, o. «}. ■ 

EXAMPLE 2 

The set 0 of odd positive integers less than 10 can be expressed by 0 = {1, 3. 5, 7, 9}. 

■ 

EXAMPLE 3 

Although sets are usually used to group togcLher elements with common properties, 
there is nothing that prevents a set from having seemingly unrelated elements. For in¬ 
stance, {a, 2, Fred, New Jersey} is the set containing the four elements a, 2, Fred, and 
New Jersey. ■ 

Uppercase letters tire usually used to denote sets. The boldface letters N, Z, and R 
will be reserved to represent the set of natural numbers (0, 1, 2, 3,.,, }, the set of integers 
{ — -2, 1,0, 1,2,..and the set of real numbers, respectively. We will occasionally 

use the notation Z + to denote the set of positive integers. (Some people do not consider 
0 a natural number, so be careful to check how the term natural numbers is used when 
you read other books.) 

Sometimes the brace notation is used to describe a set without listing all its mem¬ 
bers. Some members of the set are listed, and then ellipses (...) are used when the 
general pattern of the dements is obvious. 

EXAMPLE 4 

The set of positive integers less than 100 can be denoted by (1. 2, 3. 99}, ■ 

Since many mathematical statements assert that two differently specified collec¬ 
tions of objects are really the same set, we need to understand what it means for two 
sets to be equal. 

web 

Bertrand Russell \ 1X72-1970). Bertrand Russel! wjw bom into a prominent English family active in the 
progressive movement and having a strong commitment to liberty. He became an orphan at an early age anti 
was placed in the care of his father’s parents, who hatl him educated at home. He entered Trinity College, 
Cambridge, in 1890. where he excelled in mathematics and in moral science. He won a fellowship on the 
basis (if his work on the foundations of geometry. In 1910 Trinity College appointed him to a lectureship in 
logic and the philosophy of mathematics, 

Russell fought for progressive causes throughout his life. He held strong pacifist views, and his protests 
against World War 1 led to dismissal from his position at Trinity College. He was imprisoned for ft months 
in 1918 because of an article he wrote that was branded as seditious. Russell fought for women's suffrage 
itt Great Britain, in 1961, at the age of 89. he was imprisoned for the second time for his protests advocating 
nuclear disarmament. 

Russell's greatest work was in his development of principles that could be used as a foundation for all 
ot mathematics. His most famous work is Print ipiu Methematicu. written with Alfred North Whitehead, 
which attempt* to deduce all of mathematics using a set of primitive axioms. He wrote many books on 
philosophy, physics, and his political ideas, Russell won the Nobel Prize for literature in 1950. 
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DEFINITION 2. Two sets are equal if add only if they have the same elements. 


EXAMPLE 5 The sets }i, 3, 5}and {.l 5, if are equal, since they have the same elements. Note that 

the order in which the elements of a set are listed does not matter. Note also that it does 
not matter if an clement of a set is listed more than once, so that {J, 3. 3, 3,5,5. 5,5} is 
the same as the set {I, 3,5} since they have the same elements. ■ 


Another way to describe a set is to use set builder notation. We characterize all 
those elements in the set by staling the property or properties they must have to be 
members. For instance, the set O of all odd positive integers less than 10 can be writ¬ 
ten as 

0 {,v j r is an odd positive integer less than 10}. 

We often use this type of notation to describe sets when it is impossible to list all the 
elements of the set. For instance, the set of ail real numbers can be written as 

R - {a j x is a real number}. 

Sets can also be represented graphically using Venn diagrams, named alter the 
English mathematician John Venn, who introduced their use in 1881. in Venn diagrams 
the universal set L\ which contains ail the objects under consideration, is represented 
by a rectangle. Inside*this rectangle, circles or other geometrical figures are used to 
represent sets. Sometimes points are used to represent the particular elements of the 
set. Venn diagrams are often used to indicate the relationships between sets. We show 
how a Venn diagram can be used in the following example. 


EXAMPLE f> Draw a Venn diagram that represents V. the sc? of vowels in the English alphabet. 

Solution: We draw a rectangle to indicate the universal set U, which is the set of the 
26 letters of the English alphabet. Inside this rectangle we draw' a circle to represent V. 
Inside this circle we indicate the dements of V with points (see Figure 1). I 


We will now introduce notation used to describe membership in sets. We write 
a £ A io denote that a is an element of the set A, The notation a £ A denotes that a 
>s not a member of the set A. Note that lowercase letters are usually used to denote 
dements of sets. 


.Iiihn t cun (2 $44-1923). John Venn was born into a London suburban family noted fur its philanthropy 
web Ho attended London schools and got his mathematics degree from Cams College, Cambridge, i n 1857. He 
was elected a fellow of this college and held his fellowship there until his death. He look holy orders i n 18*9 
and, after a hriei stint of leligious work, returned to Cambridge, where be developed programs in the moral 
sciences, Besides his mathematical work, Venn had an interest in history and wrote extensively abm.it his 
college and family, 

Venn's hook Symbolic Logicclarifies ideas originally presentedhy Boole. In this book. Venn presents 
a sy sic inane development of a method that Uses geometric figures, known now as 1 -rnn diagram). Today 
these diagrams are primarily used to analyze logical arguments and to illustrate relationships between sets, 
hi addition to Jib work on symbolic logic. Venn made contributions to probability theory described in his 
widely used textbook on that subiecr. 
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FIGURE 1 Venn Diagram for the Set of FIG IRE 2 Venn Diagram Showing 
Vowels. That ,4 Is a Subset of B. 


There is a special set that has no dements. This set is tailed the empty set, or null 
set, and is denoted by 0. The empty set can also be denoted by { } (that is, we represent 
the empty set with a pair of braces that encloses all the elements in this set). Often, a 
set of elements with certain properties turns out to be the null set. For instance, the set 
of all positive integers that are greater than their squares is the null set. 

DEFINITION 3. The set A is said to be a subset of B if and only if every element 
of A is also an element of B. We use the notation A C B to indicate that A is a subset 
of the set B, 

We see that A C B if and only it the quantification 

V.vi.r S A — r <E B) 

is true. For instance, the set of all odd positive integers less than 10 is a subset of the 
set of all positive integers less than 10. The set of all computer science majors at your 
school is a subset of the set of all students at your school. 

The null set is a subset of every set, that is, 

0CS 

whenever S is a set. To establish that the null set is a subset of S, we must show that ever} 
element of the null set is also in S. In other words, we must show that the implication 
“if _r E 0, then v £ S'" is always true. We need only note that the hypothesis of this 
implication—namely, "x E 0” —is always false to see that this implication is always 
true. Hence, the empty set is a subset of every set. Furthermore, note that every set is a 
subset of itself (the reader should verify this). Consequently, if Pis a set, we know that 
0 C P and P C P. 

When we wish to emphasize that a set A is a subset of the set B but that A A B. we 
write A C B and say that A is a proper subset of B, Venn diagrams can be used to show 
that a set A is a subset of a set B, We draw the universal set 11 as a rectangle, Within 
this rectangle we draw a circle for B. Since A is a subset of B, we draw the circle for A 
within the circle for B. This relationship is shown in Figure 2. 

One way to show that two sets have the .same elements is to show that each set is a 
subset of the other. In other words, we can show that if A and B are sets with ,4 C B and 
B C A. then A = B. This turns out to be a useful way to show that two sets are equal. 

Sets may have other sets as members. For instance, we have the sets 

{0, { a\, {b}, {a, &}} and {jr | x is a subset of the set {a, b}\. 

Note that these two sets are equal. 
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Sets arc used extensively in counting problems, and for such applications we need 
to discuss the sue of sets. 

DEFINITION 4 Let 5 be a set If there are exactly n distinct elements in S where 
n is a nonnegative integer, we say that S is a finite set and that n is the cardinality of 

S. The cardinality of S is denoted by |5|. 

KX AMPLE 7 

Let .4 be the set of odd positive integers less than 10. Then Uj - 5. ■ 

EXAMPLE 8 

Let S be the set of letters in the English alphabet. Then \Y = 26, ■ 

EXAMPLE 9 

Since the null set has no elements, it follows that j 0 j - 0. ■ 

We will also be interested in acts that tire not finite, 

DEFINITION 5. A set is said to be infinite if it is not finite. 

EXAMPLE IP 

The set of positive integers is infinite ■ 

The cardinality of infinite sets will be discussed in Section 1.7. In that section, 
we will discuss what it means for a set to be countable and show that certain sets are 
countable while others are not. 

THE POWER SET 

Many problems involve testing all combinations of elements of a set to see if they satisfy 
some property To consider all such combinations of elements of a set 5. we build a new 
set that has as its members all the subsets of 5. 

DEFINITION 6, Given a set 5, the power set of $ is the set of all subsets of the 
set S. The power set of S’ is denoted by P{S). 

EXAMPLE 11 

What is the power set of the set ft). 1, 2}? 

Solution, Tlie power set P({0, i, 2}) is the set of all subsets of {0, 1, 2}. Hence. 

Pi\0, 1,2}) - {0, {(>}, {1}. {2}, {0. 1}. (0,2}. {1. 2}. {0, 1.2}}. 

Note that the empty set and the set itself are members of this set of subsets. ■ 
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EXAMPLE 12 


web 


What is the power set of the empty set? What is the power set of the set |0}? 

Solution: The empty set has exactly one subset, namely, itself. Consequently, 
P(0) = {0}. 

The set {0} has exactly two subsets, namely, 0 and the set {0} itself. Therefore, 

ram = mmi 


If a set has n elements, then its power set has 2" elements. We will demonstrate this 
fact in several ways in subsequent sections of the text, 

CARTESIAN PRODUCTS 


The order of elements in a collection is often important. Since sets are unordered, a dif¬ 
ferent structure is needed to represent ordered collections. This is provided by ordered 
n-tuples. 


DEFINITION 7. The ordered n-tuple (a \, ai ,..., o a ) is the ordered collection that 
has ai as its first dement, a--> as its second element,,,., and a„ as its nth element. 

We say that two ordered n-tuples are equal if and only if each corresponding pair 

of their elements is equal. In other words, . a n ) - (b\, b%,..b„) if andonly 

if ai - hi, for i = 1,2, — n. In particular, 2-tuples are called ordered pairs. The 
ordered pairs (a, b) and (c, d ) are equal if and only if a = c and b = d. Note that (a. h) 
ami (b, a) are not equal unless a = b. 

Many of the discrete structures we will study in later chapters are based on the 
notion of the Cartesian product of sets (named alter Rene Descartes}. We first define 
the Cartesian product of two sets. 


Rene Descartes (15*) 6-16501, Rene Descartes was born into a noble family near Tours, Fiance, about 
200 miles southwest of Paris. He wastfae third child of his father's first wife: she died several days after his 
birth. Because of Rene’s poor health, his father, a provincial judge, let his son’s formal lessons slide until, at 
the age nf 8, Rene entered the Jesuit college at La Fleche. The rector of the school took a liking to him and 
permitted him to stay in bed until late jn the morning because of his frail health. From then on, Descartes 
spent his mornings in bed; he considered these itmes his most productive hours far thinking. 

Descartes left school in 1612, moving to Pans, where he spent 2 years studying mathematics. He earned 
a law degree in 1616 from the University of Poitiers. At 18 Descartes became disgusted with studying and 
decided to see the world. He moved to Paris and became a successful gambler. However, he grew tired of 
bawdy living and moved to lire suburb of Saint-Germain, where he devoted himself to mathematical study. 
When his gambling friends found him, he decided to leave France and undertake a military career. However, 
he never did any fighting. One day, while escaping the cold in an overheated room at a military encampment, 
lie bad several feverish dreams, which revealed his future career as a mathematician and philosopher 
After ending his military' career, he traveled throughout Europe. He then spent several years in Paris, 
where he studied mathematics and philosophy and constructed optical instruments. Descartes decided to 
move to Holland, where he spent 20 years wandering around the country, accomplishing his most important 
work. During this time he wrote several books, including the Di scours, which contains his contributions to 
analytic geometry, for which he is best known. He also made fundamental contributions to philosophy 
In 1641 Descartes was invited by Queen Christina to visit her court in Sweden to tutor her in philoso¬ 
phy. Although he was reluctant to live in what he called "the land ofbears amongst rocks and ice " he finally 
accepted the invitation and moved to Sweden Unfortunately, the winter of 1649-1650 was extremely bitter, 
Descartes caught pneumonia and died in mid'February. 
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DEFINITION 8. Let A and B be sets. The Cmman pmduet of A ail & denoted 
by A x B, is the set of all ordered pairs (a, b) where a 6 A asd b € B* Heacc, 

AXB = {fof>)|ae AAbeB), 

EXAMPLE 13 

Let A represent the set of all students at a university, and let H represent the set of ail 
courses offered at the university. What is the Cartesian product A X 6? 

Solution: The Cartesian product A X B consists of all the ordered pairs of the form 
[a, b), where a is a student at the university and b is a course offered at the university. 
The set AXB can be used to represent all possible enrollments of students in courses 
at the university. ■ 

EXAMPLE 14 

What is the Cartesian product of A = {! , 2} and 8 - {«, b, r}? 

Solution: The Cartesian product AXB is 

A \ U {(1. a), fl. b\ i 1, r;. (2, «), (2, hi. i2, c)}. ■ 

The Cartesian products A x B and B X A are not equal, unless A - 0 or B A 0 
(so that A x B = 0) nr unless A = B (see Exercise 24, at the end of this section). This 
is illustrated in the following example. 

EXAMPLE 15 

Show that (he Cartesian product B X A is not equal to the Cartesian product A X B. 
where A and B are as in Example 14. 

Solution: The Cartesian product B x A is 

B ':< A - {(«. 1 j. (a, 2), (b, 1 ), (b, 2), ft. 1), fr, 2)}. 

This is not equal to A x B. which was found in Example 14 ■ 

The Cartesian product of more than two sets can also be defined. 

DEFINITION 9. The Cartesian product of the sets A], A;,..., A n , denoted by 

A[ x A 2 x ■■■ x A,, is the set of ordered n-tuples (a i, a 2 ,a„), where a,- belongs 
to A; for i = 1, 2,.... n. In other words 

A| X A 2 X X A„ - {(d], a 2 ,.. a„) \ E A t for / = l, 2, .... n}. 

EXAMPLE 16 

What is the Cartesian product A x 8 X C. where A = {0. 1), B - {1, 2}, ant! C = 
{0,1,2}? 

Solution: The Cartesian product AXBxC consists of all ordered triples (a. h, r), where 
a £ A, b G B, and c £ C. Hence, 

AXBXC - {(0,1,0), (0,1,1), (0,1,2), (0,2,0), (0,2,1), (0,2,2), (1,!, Q). (1, 1 ,1 ,t. 

(U,2),tT.2.0).(l,2. ]),(),2,2)}. ■ 
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Exercises 


1. List the members of the following sets. 

a) {a x is a real number such that jr = 11 

b) {.r j x is a positive integer less than 12} 

(i l {r | x is the square of an integer and v < 100} 
d) {.v | x is an integer such dial 4 - 2} 

2. Use set builder notation to give a description of each of 
the following sets, 

a) {0.3, 6.9,12} 

b) (-3.-■■?, -1,0. 1.2.3} 
d {m, n, o, p} 

,3. Determine whether each of the following pairs of sets 
is equal. 

a) (I, 3,3,3,5.5.5. 5, 5}. {5,3. 1} 

l» HUUi.il}} 

C) 0.{0] 

4. Suppose that A {2, 4,6}, 8 {2. 6}, C = {4,6}. and 

D - {4.6, IS}. Determine which of these sels are sub¬ 
sets of which other of these sets. 

5. For each of the following sets, determine whether 2 is 
an element of that set. 

a} {a E R | x. is an integer greater than 1} 
bl {a e R | x is the square of an integer} 
d R{2}} 

<i) {{2},P}}} 
d i{2},{2,{2}}( 
f) iii2}}} 

6. For each of the sets in Exercise 5, determine whether 
{2} is an element of that set. 

7. Determine whether each of the following slated tents is 
true or false. 

a) x E {.v} b) {if C {4 c> 14 € {a} 

d) fji-J fc {{x}} 0 C ixl f)8e {a} 

8. Use a Venn diagram to illustrate the relationship A C B 
and B C C 

9. Suppose that A. B, arid C are sets such that Ac. B and 
B C C Show' that A c C. 

if). Find two sets A and B such that A E B and A C B. 

11. What is the cardinality of each of the following sets? 
ai {«} 

bl 1>H 
d {«.{4! 
d) U {<4 {«, i4!) 

12. What is the cardinality of each of the following sets? 
a) 0 

b l {0} 

c) {0, {0}} 

d) {0. {0}, {0, {0}}} 

13. Find die power set of each of the following sets. 

a) M 

hi {a.h } 

d !0, \0)\ 


14. Can you conclude that 4 = B if A and B are two sets 
with the same power set? 

15. How many elements does each of the following sets 
have'. 1 

a) P{[a, b. {a, b}U 

b) P({0, ft, {«}. {{a}}}) 

c) P{P(0 1) 

16. Determine whether each of the following sets is the 
power set of a set. 

a) 0 

b) {0. {<?}} 

c) {0. {4 {0,«}} 

d) {0,{4{fr},{£2, fc}} 

17. Let A = 0, b, c. d} and R = {y, Find 

a) A 8 

b) B :< A 

18. What is the Cartesian product A'x B, where A is the set 
of courses offered by the mathematics department at a 
university and fJ i s the set of mathematics professors at 
this university? 

19. What is the Cartesian product A X B X C, where ,4 is 
the set of all airlines and B and C are both the set of a 11 
cities in the United States? 

20. Suppose that A x B = 0, where A and B are set s. What 
can you conclude? 

21. Let A be a set. Show that 0 x A = A X 0 ■= 0. 

22. Let .4 \a, b, 0. B = {*, >’}, and C = {0, 1}. Find 

a) A Y. B X C 

b) C x B x^A 
d C >: A x B 
d) IX BXB 

23. How many different elements does A x B have if A has 
m elements and B has n elements? 

24. Show that Ax B 8 flxA. when A and It arc nonempty 
unless A B. 

*25. Show that the ordered pair (a, b) can be defined in 
terms of sets as {{a}, {a, £}}, [Him: First show that 
{{a}, {ft, b)} = {{c} r {c, d}) if and only if a = c and 
b - d ) 

*26. In this exercise Russell’s paradox is presented. Let S 
. be the set that contains a set r ti" the set a dues not belong 
to itself, so that S ^ {r | x E a}. 
a) Show that the assumption that S is a member of ,V 
leads to a contradiction. 

h) Show that the assumption that S is not a member of 
S leads to a contradiction. 

From parts fa) and (b) it follows that the set 5 cannot 
be defined as it was. This paradox can be avoided by 
restricting the types of elements that sets can have. 

*27. Describe a procedure for listing all the subsets of a fi¬ 
nite set. 
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L5_ 

Set Operations 


INTRODUCTION 

Two sets can be combined in many different ways. For instance, starting with the set 
of mathematics majors and the set of computer science majors at your school, we can 
form the set of students who are mathematics majors or computer science majors, the 
set of students who are joint majors in mathematics and computer science, the set of all 
students not majoring in mathematics, and so on, 


DEFINITION I. Let A and B be sets. The union of the sets A and 8, denoted by 
A U B, is tiie set that contains those elements that are either in A or in B, or is both. 


An element x belongs to the union of the sets A and B if and only if v belongs to ,4 or a 
belongs to B. This tells us that 

A U B - {x | x e A v X £ 5}, 

The Venn diagram shown in Figure 1 represents the union of two sets A and B, The 
shaded area within the circle representing A or the circle representing B is the area that 
represents the union of ,4 and B. 

We will give some examples of the union of sets. 


EXAMPLE 1 


The union of the sets {1,3,5} and fl, 2,3} is the set {1,2,3,5}; that is, {1,3,5} U 
{1,2,3} = [1,2.3,5}. ■ 


EXAMPLE 2 The union of the set of all computer science majors at your school and the set of all 

mathematics majors at your school is the set of students at your school who are majoring 
either in mathematics or in computer science for in both). ■ 

DEFINITION 2, Let A and B be sets. The intersection of the sets A andfl, denoted 
by ,4 fl B, is (he set containing those elements in both A and B, 




FIGURE 1 Venn Diagram Representing FIGURE 2 Venn Diagram Representing 

the Union of A and B, the Intersection of A and B. 


PDF created with pdfFactory trial version www.pdffactorv.com 













1.5 Set Operations 47 


EXAMPLE 3 

EXAMPLE 4 

EXAMPLES 


An element x belongs to the intersection of the sets A and B if and only if * belongs 
to A and x belongs to B. This tells us that 

A fi B ~ {.v | x £ A A .r £ B}. 

The Venn diagram shown in Figure 2 represents the intersection of two sets A and B. 
The shaded area that is within both the circles representing the sets A and B is the area 
that represents the intersection of A and B. 

We give some examples of the intersection of sets. 


The intersection of the sets {1,3,5} and {1,2,3} is the set {1,3}; that is. {1,3.5} n 
{1,2.3} = {1,3}. ■ 


The intersection of the set of all computer science majors at your school and the set of 
all mathematics majors is the set of all students who are joint majors in mathematics 
and computer science. I 

DEFINITION 3. Two sets are called disjoint if their intersection is the empty set 


Let A = {1,3,5.7,9} and B - {2,4.6,8.10}. Since A n B - 0, A and B are disjoint. 


We often are interested in finding the cardinality of the union of sets. To find the 
number of elements in the union of two finite sets A and B, note that |A| + \B\ counts 
each dement that is in A but not in B or in B but not in A exactly once, and each element 
that is in both A and B exactly twice. Thus, if the number of elements that are in both A 
and B is subtracted from |A| + |Bf, elements in A fi B will be counted only once. Hence, 

' |A U B\ = [A | + |B| - |A n B\. 

The generalization of this result to unions of an arbitrary number of sets is called the 
principle of Inclusion-exclusion. The principle of inclusion-exclusion is an impor¬ 
tant technique used in the art of enumeration. We will discuss this principle and other 
counting techniques in detail in Chapters 4 and 5. 

There are other important ways to combine sets. 

DEFINITION 4. Let A and B be sets. The difference of A and B, denoted by A B, 
is die set containing those dements that are in A but not in B. The difference of A 
and Bis also called the complement of B with respect to A. 

An element x belongs to the difference of A and B if and only if x 6 A and ,v £ B. 
This tells us that 

A - B = {x j jc 6 A A x £ B). 

The Venn diagram shown in Figure 3 represents the difference of the sets A and B. The 
shaded area inside the circle that represents A and outside the circle that represents B 
is the area that represents A - B, 

We give some examples of differences of sets. 
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EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


EXAMPLE9 



FIGLRE 3 Venn Diagram for the Dif¬ 
ference of A and B, 



A i$ shaded. 

FIGURE 4 Venn Diagram for the 
Complement of the Set.l. 


The difference of {], 3,5} and {1, 2, 3} is the set {5}; that is, {!., 3,5) - {], 2,3} -= {5}. 
This is different from the difference of {1,2,3} and {1,3,5}, which is the set {2}. ■ 


Hie difference of the set of computer science majors at your school and the set of math¬ 
ematics majors at your school is the set of all computer science majors at your school 
who are not also mathematics majors. ■ 


Once the universal set V has been specified, the complement of a set can be de¬ 
fined, 

DEFINITION 5. Let V be the universal set. The complement of the set A, denoted 
by A, is the complement of A with respect to U. In other words, the complement of 
the set A is U — A. 


An element belongs to A if and only if x (£ A. This tells us that 
A = {x | x £ A}. 

In Figure 4 the shaded area outside tiie circle that represents A is the area represent¬ 
ing A. 

Wc give some examples of the complement of a set. 

Let A {a, e, i, o, u\ (where the universal set is the set of letters of the English alpha¬ 
bet). Then A = {b, c, d. f, g. h, j, k, /. m, n. p. q. r. s, t, v, ve, x, y, z). ■ 


Let A be the set of positive integers greater than 10 (with universal set the set of all 
positive integers). Then A - {1,2,3,4,5,6,7,8,9,10}. ■ 


SET IDENTITIES 


Table 1 lists the most important set identities. We will prove several of these identi¬ 
ties here, using three different methods. These methods are presented to illustrate that 
there are often many different approaches to the solution of a problem. The proofs of 
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EXAMPLE W 


TABLE 1 Set Identities, 

Identity 

1 Same 

A U 0 = A 

I Identity lawx 

A n U * A 


A U U -= V 

. 

Domination laws 

a r 0-0 


A U A - A 

Idempotent laws 

A n A - A 


(A) = A 

Complementation law 

A U B = B U A 

Commutative law's 

A P B = B 0 A 


A U (B U C) = (A Li B) U C 

Associative law's 

a n (B n n = (A n B) n c 


a n [B u o = (A n B) u (A n O 

Distributive laws 

A U \B n C) = {.A U Bi n (A U C) 


A- Fi - A n 8 

De Morgan’s laws 

A n & = A U 8 



the remaining identities will be left as exercises. The reader should note the similar¬ 
ity between these set identities and the logical equivalences discussed in Section 1.2. 
In fact, the set identities given can be proved directly from the corresponding logical 
equivalences. Furthermore, both are special cases of identities that hold for Boolean 
algebra (discussed in Chapter 9). 

One way to prove that two sets are equal is to show that one of the sets is a subset 
of the other and vice versa. We illustrate this type of proof by establishing the second 
of De Morgan’s laws. 


Prove that A O B = AUB by showing that each set is a subset of the other. 


Solution: First, suppose that v P A 0 B_ It follows that x ® A n B. This implies that 
£ A or x £ B, Hence, x £ A or x E fi. Thus, x £ A U B. This shows that A fl B C 
A U B. 

Now suppose that x £ A U B. Then x £ A or x £ B. It follows that x £ A or x £ B. 
Hence, x £ AnB. Therefore, jt £ A P. B. This demonstrates that A U S C A h fl. Since 
we have demonstrated that each set is a subset of the other, these two sets must be equal 
and the identity is proved. | 


Another way to verify set identities is to use set builder notation and the rules of 
logic. Consider the following proof of the second of De Morgan’s laws. 
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EXAMPLE II 


EXAMPLE 12 


EXAMPLE 13 


Use set builder notation and logical equivalences to show that ARB = A U B. 

Solution: The following chain of equalities provides a demonstration of this identity: 

A n B = {.t j x € A n 5} 

- {.x u £ (A 0 8))} 

- \x - ( .re A A > (- 8)} 

- {Jt I a £ A V -t £ 8} 

- {x ] x £ A V x 6 B\ 

= [x I x e A u fi). 

Note that the second De Morgan’s law for logical equivalences was used in the fourth 
equality of this chain. ■ 


Set identities can a! so be proved using membership tables. We consider each com¬ 
bination of sets that an element can belong to and verify that elements in the same com¬ 
binations of sets belong to both the sets in the identity. To indicate that an element is 
in a set. a 1 is used; to indicate that an element is not in a set, a 0 is used. (The reader 
should note the similarity between membership tables and truth tables,) 


Use a membership table to show that A f) (B U Cl - (A D 8) U (A fi C). 

Solution: The membership table for these combinations of sets is shown in Tabie 2. 
This table has eight rows. Since the columns for A n (8 U C) and (A n 8) U (A D Cl 
are the same, the identity is valid. ■ 


Additional set identities can be established using those that we have already proved. 
Consider the following example. 


Let A, 8, and C be sets. Show that 

A u t«n a - {c u fi) n A, 


TABLE 2 

A Membership Table for the Distributive Property. 


A 

fi 

c 

fiuc j a n (fi u o 

a nfi 

Anc 

(A n b ) u (A n O 
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EXAMPLE 14 



FIGURE S (a) .4 U/fuC Is Shaded, (b) .4 iflnC Is Shaded. The Union and Intersection 
of 4, B. and C. 


Solution: We have 


a u (gncj - a n(fine) 
= A n (B U C) 
= (fi U C ) n A 
■ B)' A 


by the tirsf De Morgan’s law 
by the second De Morgan's law 
by (he commutative law for intersections 
by the commutative law for unions, 


■ 


GENERALIZED UNIONS AND INTERSECTIONS 

Since unions and intersections of sets satisfy associative laws, the sets A U B u C and 
,4 H B n C are well defined when A, 5, and C are sets. Note that 4 U B U C contains 
those elements that are in at least one of the sets 4, fi, and C and that A n B fi C contains 
those elements that are in all of 4, B and C These combinations of the three sets. A, B , 
and C, are shown in Figure 5. 


Let A = {0, 2,4, 6, 8}, B = {ft 1.2. 3,4}, and C = {0, 3, 6,9}, What arc 4 U B U C and 
4 n B n C? 

Solution; The set A U B U C contains those elements in at least one of A, B, and C. 
Hence, 

4 UfiUC - {(), 1,2, 3.4,6, 8,9}. 

The set A ft B H C contains those elements in all three of 4, B, and C. Thus. 

A H B f i C = {()}. 


We can also consider unions and intersections of an arbitrary' number of sets. We 
use the following definitions. 


DEFINITION 6. The union of a collection of sets is the set that contains those 
dements that are members of at least one set in the collection. 
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EXAMPLE 15 


EXAMPLE If) 


We use the notation 

Ai U/V, U U A„ = jAj 

j 1 ■= i 

to denote the union of the sets A\,A 2 . A„. 


0EFINITION 7. The intersection of a collection of sets is the set that contains 
those elements that are members of all the sets in the collection. 

We use the notation 

A ; . ,v n • n A„ = 

i -1 

lo denote the intersection of the sets A,, Ai _A„. We illustrate generalized unions 

and intersections with the following example. 


Let A, = {i. i + 1, i + 2,...}. Then 

u* = + I. f + 2 — > = {1,2,3,...}. 

i = I f = 1 

and 

P| A, - p{i, i + l, t + 2....} = {n, n+ 1,0 + 2,,,.}. ■ 

f - L i - 1 

COMPUTER REPRESENTATION OF SETS 


There are various ways to represent sets using a computer. One method is to store the 
elements of the set in an unordered fashion. However, if this is done, the operations of 
computing the union, intersection, or difference of two sets would be time-consuming, 
since each of these operations would require a large amount of searching for elements. 
We will present a method for storing elements using an arbitrary' ordering of the ele¬ 
ments of the universal set, This method of representing sets makes computing combi¬ 
nations of sets easy. 

Assume that the universal set U is tinite (and of reasonable size so that the number 
of elements of U is not larger than the memory size of the computer being usedj. First. 

specify an arbitrary ordering of the elements of U. for instance u lf as.Represent 

a subset A of U w ith the bit string of length n, where the fth bit in this string is I if a, 
belongs to A and is 0 if a,- does not belong to A. The following example illustrates this 
technique. 


Let U - {1, 2,3, 4,5, 6,7,8, 9, 10}. and the ordering of elements of U has the elements 
in increasing order; he., a, = i. What bit strings represent the subset of all odd integers 
in U. the subset of all even integers in U. and the subset of integers not exceeding 
5 in U1 
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EXAMPLK 17 


FXAMPLf! IS 


Solution: The bit string that represents the set of odd integers in U, namely, {!, 3,5,7,9}, 
has a one bit in the first, third, fifth, seventh, and ninth positions, and a zero elsewhere, 
ft is 


10 1010 1010 , 

(We have split this bit string of length 10 into two blocks of length five for easy reading 
since long bit strings arc difficult to read.) Similarly, we represent the subset of all even 
integers in U, namely, {2,4, 6, 8,10}, by the string 

01 0101 0101 , 

lire set.of all integers in U that do not exceed 5, namely, {1, 2,3,4, 5}, is represented by 
the string 

11 1110 0000 , ■ 


Using bit strings to represent sets, it is easy taJfind complements of sets and unions, 
intersections, and differences of sets. To find the bit string for the complement of a set 
from the bit string for that set, we simply change each I to a 0 and each 0 to 1, since 
x £ A if and only if x ffi A, Note that this operation corresponds to taking the negation 
of each bit when we associate a bit with a truth value—-with 1 representing true and 0, 
false. 


We have seen that the bit string for the set {1, 3,5,7,9} (with universal set {1,2, 3, 4. 
5.6,7, S. 9, 10}) is 

10 1010 1010 . 

What is the bit string for the complement of this set 5 

Solution: The bit string for the complement of this set is obtained by replacing Os with 
Is and vice versa. This yields the string 

01 0101 0101 . 

which corresponds to the set {2,4, 6, 8, 10}. ■ 


To obtain the bit string for the union and intersection of two sets we perform bitwise 
Boolean operations on the bit strings representing the two sets. The bit in the ith position 
of the bit string of the union is 1 if either of the bits in the ith position in the two strings 
is 1 (or both are 1) and is 0 when both bits are 0, Hence, the bit string for the union 
is the bitwise OR of the bit strings for the two sets. The bit in the ith position of the 
bit string of the intersection is 1 when the bits in the corresponding position in the two 
strings are both ] and is 0 when either of the two bits is 0 (or both are). Hence, the bit 
string for the intersection is the bitwise AND of the bit strings for the two sets. 


The bit strings for the sets (I, 2. .3,4, 5} and {1,3,5. 7.9} are 11 III 0 0000 and 10 1010 
1010, respectively. Use bit strings to find the union and intersection of these sets. 
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Solution: The bit string for the union of these sets is 

u mo oooo v io loio io io = i i 1110 ioio, 

which corresponds to the set {1,2,3,4, 5, 7,9}. The hit string for the intersection of these 
sets is 

11 ! 110 0000 A 10 1010 1010 - 10 1010 OOOO, 
which corresponds to the set {1,3,5}. ■ 


Exercises 


1. Lei 4 be the set of students who live within one mile 
of school and let R be the set of students who walk to 
classes. Describe the students in each of the following 
sets. 


a) 0 fill h) A U B 

c t A - 8 d> B - A 

2. Suppose that A is the set of sophomores at your school 
and fi is the set of students in discrete mathematics at 
your school. Express each of the fol lowing sets in terms 
of A and B 

a) the set of sophomores taking discrete mathematics 
in your school 

b) the set of sophomores at your school who are not 
taking discrete mathematics 

c) the set of students at your school who either are 
sophomores or are taking discrete mathematics 

d) the set of students at your school who either are not 
sophomores or are not taking discrete mathematics 

3, Let A - {1,2. 3.4, 5} and B = {0,3,6}. Find 

a) A U R. b) A (3 B. 

ct A - H d) R - A. 


4, Lei A = {«. b, c, d, t'{ and if --- {a, b, c, d e. f, g, h]. 
Kind 

a) A U B. h) A n B 

c) A B d) B - A, 

5, Let A be a set. Show that A = A. 

6, Let A be a set. Show that 

a) A U0 = A. b) A 00 =. 0. 

Cj A UA = A. d) AHA = A. 

e) A - 0 -■ A. 0 A U U ~ U. 

g) A n If = A, hi 0 A - 0. 

7, Let A and R he sets. Show that 

a) A U B = 8 U A. b) A n B - Bn A. 

8, Find the sets A and ll if A B = [1, 5, 7, 8}, B - A - 
{2, 10}. and A D B = {3,6,9}. 

9, Show that it .4 and B are sets, then A U B ~ A fri B. 

a) by showing each side is a subset of the other side 

b) using a membership table 
10. I .et A and B be sets. Show that 

at (A n ft) c A. bi A c (A u B). 


cJA-fiCA. d) A lri (B - A) = 0, 

t) A U (B - A) = A u B. 

11. Show that if A, B, and C are sets, then .4 H B ri C - 
AUBUC 

a) by showing each side is a subset of the other side. 

b) using a membership table. 

12. Let .4, fi, and C be sets. Show that 

a) (A U B) C (A USU Cj. 

b) (A nun Q c (A D B), 

c) (A - B) - C C A " C. 

d) (A -on tc -- B) = 0. 

e> (B - A)U(C-A) = (B U C) - A. 

13. Show that if A and B arc sets, then A - B = .4 n B. 

14. Show that if A and B are sets, then (A n B) U 

(A n B) = a. 

15. Let A, 6, and C be sets. Show that 

at A U (B U C) = (A J B) U C. 

b) a n (B n c) =■ f.4 a ft . r i c 

c) .4 u i h n c) = (A u B) n (A u C). 

Iti. Let A, B. and C be sets. Show that (A - B) - C - 

(A - C) - f B - C. 

17. Let A = {0, 2,4,6, 8, 10}, B = {0. 1. 2, 3.4, 5, 6}, and 
( = {4, 5, 6, 7, 8.9,10}. Find 

a) A Iri H n C. b) A U R U C 

c) 1,4 U B) n C. d) (A n B) u C. 

18. Draw the Venn diagrams for each of the following com¬ 
binations of the sets A, B, and C, 

a) A n (B u CJ b) A n B n c 
el (A - it) U (A - Cl <J (B CJ 

19. What can you say about the sets A and B if the following 
are true? 

a) A U B =-- A b) A ft B =■ A 

e) A - B - A dj A 0 B = fin A 

e) A - B - B — A 

20. Can you conclude that A ~ B if A, R, and C are sets 
such that 

a! A UC=B U C? 

b) A n c -= fine 1 

21. Let A and fi be subsets of a universal set If. Show that 
A C B if and only if B C A. 
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The symmetric difference of A and B, denoted by A © B, 
is the set containing those elements in either ,4 or B, but not 
in both A and B 

22. Find the symmetric difference of {1, 3, 5} and {1,2,3}. 

23. Find the symmetric difference of the set of computer 
science majors at a school and the set of mathematics 
majors at this school. 

24. Draw a Venn diagram for the symmetric difference of 
the sets A and B. 

25. Show that A © B ~ fA U B) - (.4 n Bj. 

26. Show that A 9 B - (4 - B) U (B - 4). 

27. Show that if A is a subset of a universal set U. then 

a) A®4 = 0. b) 4®0 « A. 

c) A Gif = A. d) A ffi A = V. 

28. Show that if A and B are sets, then 

a) A® B = B© A. b) (A ©B) ©B = A. 

29. What canyousay about the sets A and B if A©B = A? 
*30. Determine whether the symmetric difference is asso¬ 
ciative; that is, if A, B, and C are sets, does it follow 
that 

A ® (B © C) = (4 © B) © C? 

*31. Suppose that 4, B, and C are sets such that 4 © C = 
B © C. Must Lt be the case that A - B': 

32. If A, B. C. and D are sets, does it follow that (A © B) © 
(C©D) = (A © C) 0 (B ® D)? 

33. If A. B, C. and D are sets, does it follow that (A © B) © 
«' © !>: = (A © />) © (B © C)’? 

*34. Show that if A, B. and C are sets, then 

|A U B U C\ ■--- -A\ + ,Bj + |C - |A fi B\ 

- iA n C - [B n c| + |a n b n c[. 

(This is a special case of the inclusion-exclusion prin¬ 
ciple, which will be studied in Chapter 5.) 

35. Let A; = {1,2.3.if for r = 1,2,3.Find 

a) jj A,. b) f] t. 

1=1 (=1 

36* Let A; {/, i -f I, i + Z..Find 
a) jj A,. b) P A,. 

37. Let ,4, be the set of all nonempty hit strings (that is, bit 
strings of length at least one) of length not exceeding i. 
Find 

a) (Jdi- b) 

i-i t=[ 

38. Suppose that the universal set is U ~ {1,2,3, 4, 

5,6,7, 8, 9, 10). Express each of the following sets 
with bit strings where the ith bit in the string is 1 if i is 
in the set and 0 otherwise. 

a) .{3,4,5} W {1,3,6.10} 

c) {2, 3,4, 7, a 9} 

39. Using the same universal set as in the last problem, find 
She set specified by each of the follow ing bit strings. 


a ) 11 1100 1111 b ) 010111 1000 

c) 10 0000 0001 

40. What subsets of a finite universal set do the following 
bit strings represent? 

a) the string with all zeros 

b) the string with all ones 

41. What is the bit string corresponding to the difference 
of two sets? 

42. What is the hit string corresponding to the symmetric 
difference of two sets? 

43. Show how bitwise operations on bit strings can 
be used to find the following combinations of 
A = {fl, b, c, d, <?}, B ~ {b, c, d. g, p, t, v}, C = 
{r, e, i, o, u. a; y,«}, and D = {d, e. h, i, n. o, i, ft. x, 

a) A J B b) A n B 

c) (A U D) n {B U C) d) A U B U C U D 

44. How can the union and intersection of n sets that all 
are subsets of the universal set U be found using bit 
strings? 

45. The successor of the set 4 is the set A U {A}. Find the 
successors of the following sets. 

a) {1, 2,3} b) 0 c) {0} dj {0, {0}} 

46. How many elements does the successor of a set with n 
elements have ? 

Sometimes the number of times that an element occurs in an 
unordered collection matters. Multisets are unordered col¬ 
lections of elements where an element can occur as a mem¬ 
ber more than once. The notation {mi ■«[, ttiyai, • • -, m r 
denotes die multiset with element a- ; occurring mj times, el¬ 
ement az occurring ma times, and so on. The rmmhers m,. 
i = 1,2, — r are called the multiplicities of the elements 
i - 1,2. r. 

Let P and Q be multisets. The union of the multisets P 
and Q is the multiset where the multiplicity of an element 
is the maximum of its multiplicities in P and Q. The inter¬ 
section of P and Q is the multiset where the multiplicity of 
an element is the minimum of its multiplicities in P and Q. 
Tin difference of P and Q is the multiset where the multi¬ 
plicity of an element is the multiplicity of the element in P 
less its multiplicity in Q unless this difference is negative, 
in which case the multiplicity is 0, The sum of P and Q is 
the multiset where the multiplicity of an element is the sum 
of multiplicities in P and Q. The union, intersection, and 
difference of P and Q are denoted by P U Q, P f I Q, and 
P - Q, respectively (where these operations should not be 
confused with the analogous operations for sets). The sum 
of P and Q is denoted by P + Q. 

47. Let A and B be the multisets {3 ■ a, 2 ■ h, I ■ c] and {2 ■ 
a. 3 ■ b, 4 ■ if}, respectively. Find 

a) A u B. b) A D B. c) A - B. 

d) B - A. e) A t B. 

48. Suppose that A is the multiset that has as its ele¬ 
ments the types of computer equipment needed by one 
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department of a university where the multiplicities are 
the number uf pieces of each type needed, and B is the 
analogous multiset for a second department of the uni¬ 
versity, For instance, A could be the multiset {107' per¬ 
sona] computers, 44 ■ routers, 6 ■ servers} and B could 
be the multiset {14 ■ personal computers, 6 • routers, 
2 • mainframes}. 

a) What combination of A and B represents the equip¬ 
ment (he university should buy assuming both de- 
ptirimenus use the same, equipment * 

b) What combination of A and B represents the equip¬ 
ment that will be used by both departments if both 
departments use the same equipment? 

ci What combination of A and B represents the equip¬ 
ment that the second department uses, hut the first 
department does not, if bo tit departments use the 
same equipment? 

d) What combination of A and B represents the equip¬ 
ment that the university should purchase if the de¬ 
partments do not share equipment? 

Fuzzy sets are used in artificial intelligence. Each element 
in the universal set V has a degree of membership, which 
is a real number between 0 and 1 (including 0 and /), in 
a tu/.zy set S. The fuzzy set S is denoted by listing the el¬ 
ements with rheir degrees of membership (elements with 


0 degree of membership are not listed). For instance, we 
write {0.6 Alice, 0.9 Brian, 0.4 Fred, 0,1 Oscar, 0.5 Rita} 
for the set F (of famous people} to indicate that Alice has a 
0.6 degree of membership in F, Brian has a 0.9 degree of 
membership in F. Fred has a 0.4 degree of membership in 
F, Oscar has a 0.1 degree of membership in F, and Rita has 
a 0.5 degree of membership in F(so that Brian is the most 
famous and Oscar is the least famous of these people). Also 
suppose thal A is the set of rich people with R - {0.4 Alice. 
0.8 Brian, 0.2 Fred, 0,9 Oscar, 0.7 Rita}. 

49. The complement of a fuzzy set 5 is the set 5, with the 
degree of the membership of an element in S equal to 
1 minus the degree of membership of this element in 
S. Find F (the fuzzy set of people who are not famous) 
and R (the fuzzy set of people who are not rich). 

50. The union of two fuzzy sets S and T is the fuzzy set 
S U if, where the degree of membership of an element 
in S U T is the maximum of the degrees of membership 
of this element in S and in J. Find the fuzzy set F Lj R 
of rich or famous people, 

51. The intersection of two fuzzy sets S and T is the fuzzy 
set S H T, where the degree of membership of an ele¬ 
ment in S ft T is the minimum of the degrees of mem¬ 
bership of this element in S and in T. Find the fuzzy set 
F n R of rich and famous people. 


1.6 

Functions 


INTRODUCTION 


In many instances we assign to each element of a set a particular clement of a second 
set (which may be the same as the first). For example, suppose that each student in a 
discrete mathematics class is assigned a letter grade from the set {A, B, C !), F). And 
suppose that the grades are A for Adams. C for Chon, B for Gtxxlfriend, A for Rodriguez, 
and F fur Stevens, This assignment of grades is illustrated in Figure I. 


Adam : , 



*;1 


Chou * -- 


CioodtrienU * - 


Rodriguez » 


Si tevons • 

iiiT 



« R 

• V 

• /.) 

—... ■-«• • r 

Assignment of Grades in a Discrete Mathematics Class, 
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EXAMPLE l 


EXAMPLE 2 


This assignment is an example of a function. The concept of a function is extremely 
important in discrete mathematics. Functions are used in the definition of such discrete 
structures as sequences and strings. Functions are also used to represent how long it 
takes a computer to solve problems of a given size. Recursive functions, which are 
functions defined in terms of themselves, are used throughout computer science; they 
will be studied in Chapter 3. This section reviews the basic concepts involving functions 
needed in discrete mathematics. 


DEFINITION 1. Let A and B be sets, k function f from A to B is an assig nm ent of 
exactly one dement of B lo each element of A. We write /(a) = b if b is the unique 
element offiaseigned by fte&nction/ to the element aofA. If/ is a function from 
A to B, we write f :A~* B. 

Functions are specified in many different ways. Sometimes we explicitly state the as¬ 
signments. Often we give a formula, such as f(.r) - x 1. to define a function. Other 
times we use a computer program to specify a function. 

DEFINITION 1 If/is a function from A to B, we say that A Is the domain off 
and B is the codomain off. If /(a) = b, we say that b is the image of a and a is a 
pre-image of b. The range off is the set of all images of elements of A. Also, if/ is 
a function from A to B, we say that/maps A to B. 


Figure 2 represents a function/ from A to B. 

Consider the example that began this section. Let & be the function that assigns 
a grade to a student in our discrete mathematics class. Note that G{ Adams) = A. for 
instance. The domain of G is the set {Adams, Chou, Goodfriend, Rodriguez. Stevens}, 
and the codomain is the set {A,B, C, D, F). The range of G is the set {A, B, C, F j-, because 
there are students who are assigned each grade except D. Also consider the following 
examples. 


Let/ be the functi on that assigns the last two hi ts of a bit siring of length 2 or greater lo 
that string. Then, the domain of/ is the set of all hit strings of length 2 or greater, and 
both the codomain and range are the set {00,01, 10, 11}. ■ 


Let/ be the function from Z to Z that assigns the square of an integer to this integer. 
Then, fix) = jr, where the domain off is the set of all integers, the codomain off can 
be chosen to be the set of all integers, and the range off is the set of all nonnegative 
integers that are perfect squares, namely, {0,1,4, 9,...■ 



FIGURE 2 The Function/ Maps A to 6, 
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EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 


(For students familiar with Pascal) The domain and codomain of functions are often 
specified in programming languages. For instance, the Pascal statement 

function flt>or{x: real): integer 

states that the domain of the floor function is the set of real numbers and its codoinain 
is the set of integers. ■ 


Two real-valued functions with the same domain can be added and multiplied. 

DEFINITION 3. Let/, and /? be functions from A to R, Then jfj + f 2 and f[f 2 
are also functions from A to R defined by 

(/i +MU - fx(x) + f 2 {x\ 

ifiJMx) = fiWMX). 

Note that the functions J\ + f 2 and /i/i have been defined by specifying their values 
at x in terms of the values of f\ and f 2 at x. 


Let /] and f 2 be functions from R to R such that M.xi = x 2 and fi{x) = x - x 2 . What 
are the functions /[ +- f 2 and f f 2 ! 

Solution: From the definition of the sum and product of functions, it follow s that 
l/i + /; A: - /lU) 4- fj{x 3 = X 2 + (x - x\: = x 
and 

U] fz)<x) ~ ; M x! - x 1 ■ x 4 . a 


When / is a function from a set .4 to a set B, the image of a subset of A can also be 
defined. 


DEFINITION 4. Let/ be a function from the set A to the set 8 and let S be a subset 
of A. The image of S is the subset of B that consists of the images of the elements of 
S. We denote the image of S by /(S), so that 

m ~ l/U) | x e 5}. 


Let ,4 = {<;, h, r, d. e} and B - { 1, 2,3,4} with f{a) = 2,/(b) = L/(c) - 4, f(d) = 
1. and /.{<?) - 1. The image of the subset S = {£, c,d} is the set f(S) = {1,4}. ■ 

ONE-TO-ONE AND ONTO FUNCTIONS 


Some functions have distinct images at distinct members of their domain. These func¬ 
tions are said to be one-to-one. 
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EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


DEFINITION 5, A function / is said to be me-to-om, or mjeewe, if aod oaiy if 
fix) - fty) impliesthat x - >fwaUjcondyintbedomainof/.Abactionis*aid 
to be an injection if it is one-to-one. 


Remark: A function/ is one-to-one if and only if f(x) # f(y) whenever x f y. This 
way of expressing that / is one-to-one is obtained by taking the contrapositive of the 
implication in the definition, 

We illustrate this concept by giving examples of functions that are one-to-one and 
other functions that are not one-to-one. 


Determine whether the function / from {a, b, c, dj to {1,2,3,4, 5} with f(a) = 4, 
f(b) = 5, f(c) = 1, and f(d) — 3 is one-to-one. 

Solution: The function/ is one-to-one since/ takes on different values at the four ele¬ 
ments of its domain. This is illustrated in Figure 3. ■ 


Determine whether the function f(x) - x 2 from the set of integers to the set of integers 
is one-to-one. 

Solution: The function f(x) ■--- .r 2 is not one-to-one because, for instance, /(l) - 
/(-l) = Unit 1 #-1. ■ 


Determine whether the function fix) - x + 1 is one-to-one. 

Solution: The function fix) - x + I is a one-to-one function. To demonstrate this, 
note that x + 1 / y + I when x y, ■ 


Wc now give some conditions that guarantee that a function is one-to-one, 

DEFINITION 6 . A function/ whose domain and codomain are subsets of the set 
of real nu m bers is called strictly increasing if f(x) < f(y) whenever x < y and x 
and y are in the domain of/. Similarly,/ is called strictly decreasing if fix) > f(y) 
whenever x < y and x and y are in the domain off, 
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EXAMPLE 9 


EXAMPLE 10 


EXAMPLE II 


EXAMPLE 12 



From these definitions, we see that a function that is either strictly increasing or strictly 
decreasing must be one-to-one. 

For some functions the range and the codomain are equal. That is, every member of 
the codomain is the image of some element of the domain. Functions with this property 
are called onto functions. 


DEFINITION 7, A function/ from A to £ is called onto, or surjective, if and only 
if for every element bS B there is an element a E A with /{«) = b. A function/ 
is called a surjection if it is onto. 


We now give examples of onto functions and functions that are not onto. 


Let f be the function from [a, b, c,d} to {1,2, 3} defined by /(a) = 3, fib) ^ 2. 
fir) - !, and fid) = 3. Is/an onto function 1 ’’ 

Solution: Since all three elements of the codumain are images of elements in the do¬ 
main, we see that f is onto. This is illustrated in Figure 4. ■ 


Is the function /(.v) - ,x z from the set of integers to the set of integers onto? 

Solution: The function f is not onto since there is no integer x with ,r - -1, for in¬ 
stance. ■ 


Is the function fix) - .i + I from the set of integers to the set of integers onto',’ 

Solution: This function is onto, since for every integer y there is an integer x such that 
fix) - y. To see this, note that /(a) -- y if and only if x l = y, which holds if and 
only if x - y - 1. ■ 

DEFINITION S, The function/ is a one-to-one correspondence, or a bijection, if 
it is both one-to-one and onto. 


The following examples illustrate the concept of a bijection. 


L-et/ be the function from {a , b, .e, d) to { 1.2, 3,4} with /(a) ~ 4, fib) - 2, f(c) - I , 
and fid) = 3. Is/ a bijection? 


PDF created with pdfFactory trial version www.pdffactorv.com 





1,6 Functions 61 


(u\ One-to-one. 
not onto 


(bj Onto, 
not one-to-one 


fc) One-to-one, 
anJ onto 



(d} Neither oniMO'O tie (e) Not a function 


nor onto 



Solution: The function / is one-to-one and onto. It is one-to-one since the function 
takes on distinct values. It is onto since all four elements of the codomain are images 
of elements in the domain. Hence,/ is a bisection, ■ 


Figure 5 displays four functions' where the first is one-to-one but not onto, the sec¬ 
ond is onto but not one-to-one, the third is both one-to-one and onto, and the fourth is 
neither one-to-one nor onto. The fifth correspondence in Figure 5 is not a function, since 
it sends an element to two different elements. 

Suppose that / is a function from a set ,1 to itself If A is finite, then / is one-to- 
one if and only if it is onto. (This follows from the result in Exercise 58 at the end of 
this section.) This is not necessarily the case if A is infinite (us will be shown in Sec¬ 
tion 1.7). 


EXAMPLE U 1 ,et A. be a set. The identify function on A is the function ; ,4 -> .4 where 

MW x, 

where x e A. In other words, the identity function i A is the function that assigns each 
element, to itself. The function l a is one-to-one and onto, so that it is a bijection. ■ 

INVERSE FUNCTIONS AND COMPOSITIONS OF FUNCTIONS 


Now consider a one-to-one correspondence / from the set A to the set B Since / is 
an onto function, every dement of B is the image of some dement in A. Furthermore, 
because/ is also a one-to-one function, every' element of B is the image of a unique 
element of A, Consequently, we can define a new function from B to A that reverses the 
correspondence given by/. This leads to the following definition. 

DEFINITION 9, Let / be a one-to-one correspondence from the set A to the set B. 
The inverse function of / is the function that assigns to an element b belonging to B 
the unique element a in A such that f(a) - b. The inverse function of/ is denoted 
by f~ 1 . Hence, /" 1 (fb) = a when f(a) - b. 


Figure 6 illustrates the concept of a an inverse function. 

If a function / is not a one-to-one correspondence, we cannot define an inverse 
function of/. When/ is not a one-to-one correspondence, either it is not one-to-one or it 
is not onto. Iff is not one-to-one, some element b in the codomain is the image of more 
than one element in the domain. If f is not onto, for some element b in the codomain, 
no element a in the domain exists for which /(«) - /-. Consequently, if / is not a 
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EXAMPLE 14 


EXAMPLE IS 


EXAMPLE 16 





FIGURE 6 The Func tion / 1 Is the Inverse of Function f. 

one-to-one correspondence, we cannot assign to each element, b in the codomain a 
unique element a in Ibe domain such that f(a) = b (because for some b there is ei¬ 
ther more than one such a or no such a). 

A one-to-one correspondence is called invertible since we can define an inverse 
of this function. A function is not invertible if it is not a one-to-one correspondence, 
since the inverse of such a function does not exist. 


Let / be the function from {a, b. c} to {1,2,3} such that f(a') = 2. f(b) = 3, and 
/(cl = 1, Is/ invertible, and if it is, what is its inverse? 

Solution: The function / is invertible since it is a one-to-one correspondence. The 
inverse function f~ l reverses the correspondence given by /, so that f 1 (1 > = c. 
f-\l) - u.and’r'CI) - A ■ 


Let f be the function from the set of integers to the set of integers such that f(x) — x + L 
Is/ invertible, and if it is. what is its inverse? 

Solution: The function/ has an inverse since it is a one-to-one correspondence, as we 
have shown. To reverse the correspondence, suppose that y is the image of v, so that 
y - x + I. Then x = y - 1. This means that y - I is the unique element of Z that is 
sent to y by/. Consequently, f Ty) = y - I, ■ 


Let/ be the function from Z to Z with fLvi = ,r, Is/ invertible ? 

Solution: Since ft 1t = /( I ) - 1,/ is not one-to-one. If an inverse function were 
defined, it would have to assign two elements to 1. Hence,/is not invertible. ■ 

DEFINITION 10, Lei g be a function from the set A to the set B and let/ be a 
function from the set B to the set C. The composition of the functions/ and g, denoted 
by fog, is defined by 

( fogXa) ■ f(gm 


In other words, f o g is the function that assigns to die element a of A the element 
assigned by / to g(a). Note that the composition f o g cannot be defined unless the 
range of g is a subset of the domain of /. In Figure 7 the composition of functions is 
shown. 


PDF created with pdfFactory trial version www.pdffactorv.com 






1.6 Functions 63 


EXAMPLE 17 


EXAMPLE 18 


f°s 



Let,? be the function from the set {a, k c} to itself such that g(a) - b, g (b) = c, 
and g(r) - a. Let / be the function from the set {a, b, ■r} to the set {!, 2,3} such that 
/(a) - 3, / (b) = 2, and /(c) = 1. What is the composition of / and g, and what is 
the composition of g and/? 

Solution: The composition / o g is defined by (/ 0 g)(a) = f(g(a)} = fit) = 2, 
(fog) (, b) = f(g(b)) - f(c) = l,and(/og)(c) - /(g(c)) = /(a) = 3. 

Note that g 0 / is not defined, because the range of f is not a subset of the domain 
ofg- « 


Let f and g be the functions from the set of integers to the set of integers defined by 
f(x) = 2x + 3 and g{x) = 3x + 2. What is the composition off and g? What is the 
composition ofg and/"? 

Solution: Both the compositions / ogandgo/are defined. Moreover, 

(fog)(x) f(g(x )) - /(3a- + 2) - 2(3 x + 2) + 3 = bx + 7 
and 

0? ° f)(x) - g(f(x)) = g{2x + 3) - 3(2.v + 3) + 2 = 6x + li B 

Remark: Note that even though f o g and gof are defined for the functions/ and g in 
Example 18, / o g and g o f are not equal. In other words, the commutative law does 
not hold for the composition of functions. 

When the composition of a function and its inverse is formed, in either order, an 
identity function is obtained. To see this, suppose that/is a one-to-one correspondence 
from the set A to the set B, Then the inverse function / -l exists and is a one-to-one 
correspondence from B to A. The inverse function reverses the correspondence of the 
original function, so that f~H_b) =■ a when fia) = b. and J(a) = h when f^'ih) = a. 
Hence, 

ir'ofm = r\f(a)) - /->> - 

and 

ifof l m - ) = m •• b. 
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EXAMPLE 19 


EXAMPLE 20 


Consequently/' [ °f = Grand /o /" 1 - i B . where (4 and l b are the identity func¬ 
tions on the sets ,4 and ti , respectively. That is. (./ 1 )" ] ~ f. 


THE GRAPHS OF FUNCTIONS 


We can associate a set of pairs in .4 X B to each function from A to B. This set of pairs is 
called the graph of the function and is often displayed pieiorially to aid in understanding 
the behavior of the function. 


DEFINITION 11. Let/ be a function from the set A to the set B. The graph of the 
function / is die set of ordered pairs {(a, i») j a 6 A and /(«) ~ ft}. 


From the definition, the graph of a function/ from A to B is the subset of A X B containing 
the ordered pairs with the second entry equal to the element of B assigned by / to the 
first entry'. 


Display the graph of the function fin) = 2/1 - I from the set of integers to the set of 
integers. 

Solution: The graph off is the set of ordered pairs of the form (n, In + 1) where n is 
an integer. This graph is displayed in Figure 8 . ■ 


Display the graph of the function fix) = x 2 from the set of integers to the set of inte¬ 
gers. 

Solution: The graph of / is the set of ordered pairs of the form (x, f(x )) = (*, a 2 ) 
where x is an integer. This graph is displayed in Figure 9. ■ 



0 • d 0 c o 0 o 

k : - J C 1 -J U C D Q 

FIGURES The Graph or the 
Function/(n) = In 1 1 from ZtoZ. 




• (- 2 , 4 ) 

( 2 , 4 )* 

(- 1 , 1 )* 

• ( 1 , 1 ) 


( 0 , 0 ) 


FIGURE 9 The Graph of /(x) = jr 
from Z to Z. 
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EXAMPLE 21 


SOME IMPORTANT FUNCTIONS 


Next, we introduce two important functions in discrete mathematics, namely, the floor 
and ceiling functions. Let r be a real number. The floor function rounds v down to 
the closest integer less than or equal to x, and the ceiling function rounds jr up to the 
closest integer greater than or equal to a. These functions are often used when objects 
are counted. They play an important role in the analysis of the number of steps used by 
procedures to solve problems of a particular size. 


DEFINITION 12. The floor function assigns to the real number x the largest 
integer that is less than or equal to x. The value of the floor function at x is de¬ 
noted by |xj. The ceiling function assigns to the real number x the smallest integer 
that is greater than or equal to ,r. The value of the ceiling function at x is denoted 

M4 


Remark: The floor function is often also called the greatest integer function. Jt is often 
denoted by [ vj. 


'the following are some values of the floor and ceiling functions: 

l 4J = 0, rjl=L [-4j = -i. [-*1 = 0. 

■;3.1J = 3, [3.11 = 4, ,7| = 7. [ 7 ] - 7. 


We display the graphs of the floor and celling functions in Figure 10. 

The floor and ceiling functions are useful in a wide variety of applications, includ¬ 
ing those involving data storage and data transmission. Consider the following exam¬ 
ples, typical of basic calculations done when database and data communications prob¬ 
lems are studied. 



FK.I RL 1!) Graphs of the (a) Floor and (b) Ceiling Functions, 
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EXAMPLE-: 22 


EXAMPLE 23 


Data stored on a computer disk or transmitted over a data network are usually repre¬ 
sented as a string of bytes. Each byte is made tip of 8 bits. How many bytes are required 
to encode 100 bits of data? 

Solution; To determine the number of bytes needed, we determine the smallest integer 
that is at least as large as the quotient when 100 is divided by 8, the number of bits in 
a byte. Consequently, f 100/8] = 1 12.5] - 13 bytes is required. ■ 


in asynchronous transfer mode (ATM) (a communications protocol used m backbone 
networks), data are organized into cells of 53 bytes. How many ATM cells can be trans¬ 
mitted in 1 minute over a connection that transmits data at the rate of 500 kilobits per 
second? 

Solution: In 1 minute, this connection am transmit 500,000 - 60 - 30,000,000 bits. 
Each ATM cell is 53 bytes long, which means that it is 53*8 = 424 bits Jong. To deter¬ 
mine the number of cells that can be transmitted in 1 minute, we determine the largest 
integer not exceeding the quotient when 3Q,(XK),000 is divided by 424. Consequently, 
i r 30,GOO, 000/4241 = 70,754 ATM cells can be transmitted in 1 minute over a 500 kilobit 
per second connection. ■ 


Table 1 , with x denoting a real number, displays some simple but important prop¬ 
erties of the floor and ceiling functions. Since these functions appear so frequently in 
discrete mathematics, it is useful to look over these identities. Each property in this ta¬ 
ble can be established using the definitions of the floor and ceiling functions. Properties 
(la), (1 b), (k), and (Id) follow directly from these definitions. For example, (la) states 
that (jj. = n if and only if the integer n is less than or equal to .x and ;i + 1 is larger 
than x. This is precisely what it means for n to be the greatest integer not exceeding 
r, which is the definition of [a] ^ n. Properties (lb), (1c), and (Id) can be established 
similarly. 

We will show' that (4a) is true. To show that (4a) is true, suppose that [x\ = n where 
n is an integer. By (1 a) it follow's that n =£ .t < n + 1. Adding m to this inequality shows 
thatn+m ^ x+m < n +■1. Using (la) again, we see that [jcTtfiJ - n-f-.m -- [x\+m t 


TAB Ll 

1 Use hi 

Properties of the Floor and Celling Functions. 

1 la) [,v. 

= n if and 

only if n ^ 

x < n + 

1 where n is an integer 

Ub) fr 

- n if and 

only if n - 

< X 5 

n where n is an integer 

firj (y 

= rr if and 

only if x - 

< ft sr 

r where n is an integer 

(Id) \x 

= n if and 

only if x s 

rt < x + 

1 where n is an integer 

(Ike - 

1 < UJ s 

x - W < X 

+ 1 


(?*) 

tl - -[*1 




Dhi;"- 

tl = -tar] 




l4a )[x 

f mj = t-vj 

t m when m 

is an integer 

[4bjj.r 

* m, - \x) 

+■ m when m 

is an integer 
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which is what we wanted to show. We defer establishing the other' properties to the 
exercises. 

There are cert ai n types of functions that wifi be used throughout the text. These in¬ 
clude polynomial, logarithmic, and exponential functions. A brief review of the prop¬ 
erties of these functions needed in this text is given in Appendix 1. In this book the 
notation log x will be used to denote the logarithm to the base 2 of x, since 2 is the base 
that we will usually use for logarithms. We will denote logarithms to the base h , where 
b is any real number greater than 1 , by fog (l x. 


Exercises 


1 . Why is / not a function from R to R in the following 

equations'? 

a) /(*} = i/x b) fix ) - Jx 

0 f(x) = tj{x 2 + 1) 

2. Determine whether/ is a function from Z to R if 

a) f(n) = b) /(«) = fn 2 + 1. 

c) fin) = V(n 2 - 4). 

3. Determine whether/ is a function from the set of all bit 

strings to the set of integers if 

at f(S) is the position of a 0 bit in S. 

b) f(S) is the number of ! bits in S. 

c) f($) is the smallest integer t such that the ith bit of 
S is t and f(S) = 0 when S is the empty string, the 
string with no bits. 

4. Find the domain and range of the folio wing functions. 

a) the function that assigns to each nonnegative inte¬ 
ger its last digit 

b) the function that assigns the next largest integer lo 
a positive integer 

c) the function that assigns to a bit string the number 
of one bits in the string 

d) the function that assigns to a bit string the number 
of bits in the string 

5. Find the domain and range of the following functions. 

a) the function that assigns to each bit string the differ¬ 
ence between the number of ones and the number 
of zeros 

b) the function that assigns to each bit string twice the 
number of zeros in that string 

c) the function that assigns the number of bits left over 
when a bit string is split into bytes (which are blocks 
of 8 bits) 

d) the function that assigns to each positive integer the 
largest perfect square not exceeding this integer 

6 . Find (he follow ing values. 

a)Ll-U b) fl H c) L-aiJ 

d) [-0.1] e) [2,991 f) [-2.991 

BflWilJ bMLil + FM 


7. Find the following values. 

a) f|l b) H\ c) f-Jl d) l-|j 

e)f3l f)HJ «Ui+r|jJ hUHiii 

8 . Determine whether each of the following functions 

from {a, h, c.d) to itself is one-to-one. 

a) f{a) * b, f{b) = a, /(c) = c, fid) = d 

b) fid) - b, f{b) = h, /(c) = d. /(d) = c 

c) f(a) = d, fib) = b, f(c) = c, fid) = d 

9. Which functions in Exercise 8 are onto? 

10. Determine whether each of the following functions 
from Z to Z is one-to-one. 

a) f(n) = n - 1 b) f(n) = n z + I 
c) fin) = rr d) fin) = fn/ 2 ] 

11. Which functions in Exercise 10 are onto? 

12. Give an example of a function from N to N that is 

a) one-to-one but not onto. 

b) onto but not one-to-one. 

c) both onto and one-to-one (but different from the 
identity function). 

d) neither one-to-one nor onto. 

13. Give an explicit formula for a function from the set of 
integers to the set of positive integers that is 

a) one-to-one, but not onto. 

b) onto, but not one-to-one. 

c) one-to-one and onto. 

d) neither one-to-one nor onto. 

14. Determine whether each of the following functions is a 
hijeetion from R to R, 

a) fix) = -3x + 4 

b) f(x) = -3.x 1 + 7 

c) m = l x+!)/(* +2) 

d) fix) = + 1 

15. Determine whether each of the following functions is a 
btjeetion from R to R. 

a) fix) = lx + 1 

b) fix) = x 2 + 1 
C) fix) - I 5 

d) fix I = (.r +■ 1 i/t+ 2} 


PDF created with pdfFactory trial version www.pdffactorv.com 





68 1 ! The Foundations: Logic, Sets, and Functions 


16, Let S = 1-1.112,4.71. Find AS) if 

at /(f) - 1. b) fix) = 2.x +■ I. 

c) f(x) = 7v/5}. dj fix) = Lt.r +■ 1 :i/3J. 

11. Let fix) - M-V3J. Find f(S) it' 

a) ,V - { 2. MU 2, 3), b) S = {H I, 2,3,4,5/ 

cl ,9 - {1.5, 7, 11}. d> S = {2,6. 10,14}. 

IS, Let /(n 2.v, What is 

a) /( Z}1 b> f(S)l c} /(R>? 

19, Suppose that g is a function from A to B and/ is a func¬ 
tion front ff to £'. 

a) Show that if both/ and g are one-to-one functions, 
then f o g is also one-to-one. 
bl Show that if both / and g are onto functions, then 
/ o g is also onto. 

*20. r i f and / S g are one-to-one, does it follow that,? is 
one-to-one' 1 Justifv your answer. 

*21. It’ / and f 0 g are onto, does it follow that g is onto? 
Justify your answer. 

22 . Find f&g and gof where fix) = x 1 v I and g(x) - 
-V \ 2 tire functions from R to R. 

23. Find / + g and/? for the functions / and g given in 
Kucreise 22 

24. I. cl /(t) - ax t b and g (x) - cx+d where a. b,c, and 
d are constants. Determine for which constants a. b , t:. 
and i d it is into that f o g = g a f. 

25. Show that the function fix) ~ ax f h front R to R is 
invertible, where a and b are constants, with a # 0, and 
find the inverse of/'. 

26. Let/ he a f unction from the set A to the set B Let $ and 
7 he subsets of A. Show that 

a) /(Sun - /{S)U/(T}. 

b) f(SfiT) c /<>') n f {T). 

27. Uive an example to show that the inclusion in part (b) 
in Fxernse 26 may be proper. 

Let / he a function from the set A to the set B. Let ,V he a 
subset of B. We define the inverse image of 5 to be the 
subset of 4 containing all pre-images of all elements of 
S. We denote the inverse image of S by so that 

f 't/) ---■ £ A | f(n) £ .V}." 

28. Let / be the function from R to R defined by /(.v) 

,r . Find 

at / ''({Hi. b) ./-'({/()< jr< I}f. 

tl f /{a | x > 4}>. 

29. Let g{x) - t.yi. Find 

at g /(Of i. b) g J {;{-!.{), 1}). 

c) g '({.tJK x < !}f, 

30. Let/ be a function from A to B. Let S and T be subsets 
of#. Show that 

a) / '(S\i7) , / '(Sju/'/n 

bl / 1 f.Vn T) - /"'(Sin f-'iT). 

31. 1 ■el/ be a t unction front A to B L et S be a subset of B. 
Show that. / 1 {.V) 

32. Show that i.v ■ J j is the closest integer to the integer ,v. 
except when ,r is midway between two integers, when 
it is the larger of these two integers, 


33. Show 1 that |a - j} is the closest integer to the integer x. 
except when r is midway between two integers, when 
it is the smaller of these two integers. 

34. Show that if x is a real number, then [.ti -- tj = 1 if .v 
is not an integer and /if - [_r| = 0 if x is an integer. 

35. Show' that if t is a real number, then r 1 < }.v 

x s }.({ < x -t 1 . 

36. Show that il\r is a real number and m is an integer, then 
[.¥ -+ ml = f.rl + m. 

37. Show that if x is a real number and n is an integer, then 

a) .t < a if and oniy :r..v] < n 

b) n < a if and only if n < }a}. 

38. Show that if x is a real number and n is an integer, then 

a) ,t £ « if and only iff./ £ n. 

b) n £ jr if and only if n £ j/J. 

39. Lef jr be a real number. Show that-jLyj - |jtj -h {v * 

46. Prove that if x is a real number, then}- x\ ~ | jr| and 

f-.r| = - [ vj. 

41, The function LNT is found on some calculators, where 
IN I'D) = : rj when v is a non negative real number and 
INT(.t) - fa] when .t is a negative real numher. Show 
that this ZNTfunction satisfies the identity INT( -.t) = 
-INT(w). 

42, Let u and h be real numbers with a < h Use the hour 
and/or ceding functions to express the number of inte¬ 
gers n that satisfy the inequality -i - n - » 

43, L.ct a and b be real numbers with a < b. Use the floor 
and/or ceiling fimetions to express the number of inte¬ 
gers n that satisfy the inequality a < n < b 

44, How many bytes are required to encode n bits of data 
where n equals 

a) 4? lit 10? ci 500? di 3000? 

45, How many bytes are required to encode n bits of data 
where n equals 

a) 7? b) 17? c) 1001? d) 28800? 

46, How many ATM cells (described in Example 23) can 
be transmitted in 10 seconds over a link operating at 
the following rates? 

a) 128 kilobits per second (1 kilobit = 1000 bits) 

b) 300 kilobits per second 

e) i megabit per second (I megabit = 1,000.000 bits) 

47, Data are transmitted over a particular Ethernet network 
in blocks of 15TO octets (blocks of 8 bits). How many 
blocks are required to transmit the following amounts 
of data over this Ethernet network 11 (Note that a byte is 
a synonym for an octet, a kilobyte is 1000 bytes, and a 
megabyte is 1,000,000bytes.) 

a) 150 kilobytes of data 

b) 384 kilobytes of data 

e) 1,544 megabytes of data 
d) 45.3 megabytes of data 

48, Draw the graph of the function fin) ~ 1 - tf from Z 
to Z. 

49, Draw the graph of the function fix) }2.tj from R 
to R, 


PDF created with pdfFactory trial version www.pdffactorv.com 




1.7 Sequences and Summations 69 


50. Draw the graph of the function fix) = [x!2\ from B 
to R. 

51. Draw' the graph of the function fix) = [,rj + [.tV2j from 
R to R. 

52. Draw the graph of the function /(.r) = f.v] + [x/Z\ from 

R to R 

53. Draw graphs of each of the following functions. 

at fix) ■- L-v - ‘j bi f(x) = [2x ■+ IJ 

e) fix) - [xOT d) f(x) = fl/xl 

e) f{x) = [x - 2} -i [.r + 2j ft fix) -- |2.rj[.(/2] 

8) fix J fir - ;J + il 

54. Draw graphs of each of the following functions, 

a) fix) = [3,v - 21 b) fix) - [0.2*1 

e) fix) = [-■ \ix\ d) 'fix) - 'a- 2 ] 

e) fix) = r.«/2][.i/2J ft 'fix) - : </2j + f.v/2J 

g> fix) - |2f.v/2j r j[ 

55. Find the inverse function of f(x) = ,/ 1, 

56. Suppose that f is an invertible function from Y to Z and 
j? is an invertible function from X to Y. Show that the 
inverse of the composition fog is given by if o-tjl 1 -= 
g' ] of l . 

57. Let S he a subset of a universal set U. The charac* 
(eristic function f s of .S' is the function from V to the 
set {0. Ij such that fix) = 1 if x belongs to 5 and 
fix) ~ 0 if x does not belong to S Let A and B be 
sets. Show that for all x 

a) '• - fl*)' fsix) 

b) f.-tuBix) ~ fix) + Jfjst.l') - _/;.■!(-V> ■ f B [X) 
el /.,('.*> -- 1 - fix) 

d) fA-ml) = f A ix) + Ml) - 2f A [x)fix) 

58. Suppose that/ is a function from A to B. where .3 and B 
arc finite sets with L4| = | Bj. Show that/ is one-to-one 
if and only if it is onto. 

A program designed lo evaluate a function may not pro¬ 
duce the correct value of the function for all elements in the 


domain of this function. For example, a program may not 
produce a correct value because evaluating the function 
may lead to an infinite loop or an overflow. 

To study such situations, we use the concept of a partia! 
function, A partial function / from a set A to a set B is 
an assignment to each element a in a subset of A. called 
the domain of definition of/, of a unique dement b in B. 
Die sets A and B are called the domain and codomain of 
/, respectively. We say that/ is undefined for dements in 
A that are not in the domain of definition of f. We write 
f : A —> R to denote that/ is a partial function from A to 
B. (This is the same notation as is used for functions. The 
context in which the notation is used determines whether f 
is a partial function or a total function.) When the domain 
of definition of/ equals A, we say that/ is a total function, 

59, For each of the following partial functions, determine 
its domain, codomain, domain of definition, and the set 
of values for which it is undefined. Also, determine 
whether it is a total function. 


a) 

f 

Z 

R, /In) - 1 In 


b) 

/ 

r. 

Z, fin) = [n/2l 


o) 

/ 

z 

XZ-Q, fim, n) 

= min 

d) 

f 

z 

xZ-Z. fim, n) 

mn 

e) 

f 

z 

x Z Z. fim, it) = 

m - 


60. a) Show that a partial function from 3 to B can be 
viewed as a function f from A to B U {4 where 
u is not an clement of B and 

| /(a) if a belongs to the domain 
/'(a) =■ I of definition of/ 

[ u if / is undefined at a 

h; Using the construction in i.aj. find the function f 
corresponding to each partial function in Exercise 
59. 


\. 1 _ _ 

Sequences and Summations 

INTRODUCTION 


Sequences are used to represent ordered lists of elements. Sequences are used in discrete 
mathematics in many ways. They can be used to represent solutions to certain counting 
problems, as we will see in Chapter 5. They are also an important data structure in 
computer science. This section contains a review of the concept of a function, as well 
as the notation used to represent sequences and sums of terms of sequences. 

Whet- the elements of an infinite set can be listed, the set is called countable, We 
will conclude this section with a discussion of both countable and uncountable sets. 

SEQUENCES 


A sequence is a discrete structure used to represent an ordered list. 
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We use the notation {a„} to describe the sequence. (Note that a n represents an in¬ 
dividual term of the sequence {a n }- Also note that the notation {a n \ for a sequence con¬ 
flicts with the notation for a set. However, the context in which we use this notation 
will always make it clear when we are dealing with sets and when we are dealing with 
sequences.) 

We describe sequences by listing the terms of the sequence in order of increasing 
subscripts. 

EXAMPLE 1 

Consider the sequence {a*}, where 

a n - 1 In. 

The list of the terms of this sequence, beginning with , namely, 

Q\,a2, ai.fl*, 

starts with 

1,1,1,1... . 

’2 3 4. a 

EXAMPLE 2 

Consider the sequence {b n } with b„ = (-If. The list of the terms of this sequence, 
bn, b\, b 2 , K... , begins with 

1,-1,1,-1,1. B 

EXAMPLE 3 

Consider the sequence c a = 5". The list of the terms of the sequence c 0j c h c 2 , c 3 , c 4 , 
c 5 , .. . begins with 

1,5,25,125,625,3125. ■ 

Sequences of the form 

Q], «2, . . 

are often used in computer science. These finite sequences are also called strings. This 
string is also denoted by (jui 2 •' ‘ a n . (Recall that bit strings, which are finite sequences 
of bits, were introduced in Section 1.1.) The length of the string S is the number of 
terms in this string. The empty string is the string that has no terms. The empty string 
has length zero. 

EXAMPLE 4 

The string abed is a string of length four. B 

SPECIAL INTEGER SEQUENCES 

A common problem in discrete mathematics is finding a foimula or a general rule for 
constructing the terms of a sequence. Sometimes only a few terms of a sequence solving 
a problem are known; the goal is to identify the sequence. Even though the initial terras 
of a sequence do not determine the entire sequence (after all, there are infinitely many 
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EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


different sequences that start with any finite set of initial terms), knowing the first few' 
terms may help you make an educated conjecture about the identity of your sequence. 
Once you have made this conjecture, you can try to verify that you have the correct 
sequence. 

When trying to deduce a possible formula or rale for the terms of a sequence from 
the initial terms, try to find a pattern in these terms. You might also see whether you 
can determine how a term might have been produced from those preceding it. There 
are many questions you could a$k, but some of the more useful are; 

■ Are there runs of the same value? 

■ Are terms obtained from previous terms by adding the same amount or an amount 
that depends on the position in the sequence? 

■ Are terms obtained from previous terms by multiplying by a particular amount? 

■ Are terms obtained by combining previous terms in a certain way? 

The following examples illustrate how this type of problem might be attacked. 


What is a rule that can produce the terms of a sequence if the first 10 terms are 1. 2.2. 
3,3,3,4,4,4,4? 

Solution: Note that the integer 1 appears once, the integer 2 appears twice, the integer 
3 appears three times, and the integer 4 appears four times. A reasonable rule for gen¬ 
erating this sequence is that the integer n appears exactly n times, so the next five times 
of the sequence would all be 5, the following six terms would all be 6 , and so on. The 
sequence generated this way is a possible match. ■ 


What is a rule that can produce the terms of a sequence if the first 10 terms are 5, 1 1 , 
17,23,29,35,41,47, 53. 59? 

Solution: Note that each of the first 10 terms of this sequence after the first is obtained 
by adding 6 to the previous term. (We could see this by noticing that the difference 
between consecutive terms is 6 .) Consequently, the nth term could be produced by 
starting with 5 and adding 6 a total of n - 1 limes; that is, a reasonable guess is that the 
nth term is 5 + tin - 1) = 6 n - 1 ■ 


The sequence iir the solution of Example 6 is an arithmetic progression, which 
is a sequence of the form a, a + d.a + 2d, a -r 3d, .,., a + nd, .... In particular, this 
sequence has a - 5 and d - 6 , 

Another useful technique for finding a rule for generating the terms of a sequence 
is to compare the terms of a sequence of interest with the terms of a well-known integer 
sequence, such as terms of an arithmetic progression, terms of a geometric progression 
(see Example 12). perfect squares, perfect cubes, and so on. The first 10 terms of some 
sequences you may want to keep in mind are displayed in Table 1. 


Conjecture a simple formula for a n if the first 10 terms of the sequence {a n } are 1 , 7, 
25, 79, 34 1 , 727, 2185. 6559, 1 9681 . 59047, 

Solution: To attack this problem, wc begin by looking at the difference of consecutive 
terms, but we dc> not see a pattern, When we form the ratio of consecutive terms to see 
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TABLE 1 

Some Useful Sequences. 

nth Term 

First 10 Terms 

n 1 

1,4,9,16.25,36,49.64,81,100. ... 

n'• 

1,8, 27,64, 125, 21 6. 343, 512. 729, 1000, ... 

n 4 

1, 1ft, 81, 256, 625, 12%, 2401, 4096, 6561, 10000, ... 


2,4, 8,16, 32,64, 128,256.512, 1024, ... 

r- 

3, 9, 27, 81, 243, 729. 2187, 6561, 19683, 59049, .. 

_j 

1,2, 6, 24,120, 720, 5040,40320. 362880, 3628800, .,, 


whether each term is a multiple of the previous term, we find that this ratio, although not 
a constant, is close to 3. So it is reasonable to suspect that the terms of this sequence are 
generated by a formula involving Y. Comparing these terms with the corresponding 
terms of the sequence [3"}, we notice that the nth lerm is 2 less than the corresponding 
power of 3. We see that a n = 3" - 2 for 1 st n m 10 and conjecture- that this formula 
holds for all n, ■ 


We wall see throughout this text that integer sequences appear in a wide range of 
contexts in discrete mathematics. Sequences we will encounter include the sequence 
of prime numbers (Chapter 2), the number of ways to order n discrete objects (Chapter 
4), the number of the moves required to solve the famous Tower uf Hanoi puzzle with 
n disks (Chapter 5), the number of rabbits on an island after n months (Chapter 5), and 
the number of comparisons needed to sort n numbers (Chapter 8). 

Integer sequences appear in a fabulously wide range of subject areas besides dis- 
Wfh crete mathematics, including biology, physics, engineering, chemistry, and physics, as 
well as in puzzles. A wonderfully diverse collection of over 8000 different integer se¬ 
quences has been constructed over the past 20 years by the mathematician TSieil Sbatie. 
who has teamed up with Simon Plouffe, to produce The Encyclopedia of 1 ntcgem Se¬ 
quences ([SIP1951). An extended list of the sequences is available on the Web, with new 
sequences added regularly. There is also a program accessible via the Web that yon can 
use to find sequences from the encyclopedia that match initial terms you provide. 


Neil Sloane (born 1939). Neil Sloanc studied mathematics and electrical engineering at the University 
M >l rfy. of Melbourne on a scholarship from the Australian state telephone company. He mastered many telephone- 

related jobs, such as erecting telephone poles, in his summer work. After graduating, lie designed minimal 
cost telephone networks in Australia. In 1962 he came to the United States and studied electrical engineering 
at Cornell University. His Ph.D, thesis was on what are now called neural networks. He took a job at Bell 
Labs in 1969, working in many areas, including network design, coding theory, and sphere packing, He 
now works for AT&T Labs, moving there from Bdi Labs when AT&T split up in 1996. One of his favorite 
problems is the kissing problem fa name he coined), which asks how many spheres can be arranged in n 
dimensions so that they all touch a centra! sphere of the same size, (In two dimensions the answer is 6, since 
6 pennies can be placed so they touch a central penny. In three dimensions, 12 billiard balls can be placed 
so that they touch a centra] billiard ball. Two billiard balls that just touch arc said to ki:,v giving rise to the 
terminology "k rising problem" and ■'kissing number.") Sloanc, together with Andrew Odlyzko, showed that 
in 8 and 24 dimensions the optimal kissing numbers are, respectively, 240 and 196,560. The kissing number 
is known in dimensions 1, 2, 2. S, and 24, hut not in any other dimensions. Sloane’s books include Sphere 
Packing, r, lattices and Groups, 3d ed„ with John Conway; The Theory of Error-Correcting Codes with 
Jessie Mac Williams; The Encyclopedia of integer Sequences with Simon Plouffe; and The Rock-Climbing 
Guide to New Jersey Crags with Paul Nick. The last book demonstrates his interest in rock climbing; it 
includes more than 50 climbing sites in New Jersey. 
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SUMMATIONS 

Next, we introduce summation notation. We begin by describing the notation used to 
express the sum of the terms 

a m , a mT 

from the sequence {a,,}. Wc use the notation 

i a i 

to represent 

Here, the variable j is called the index of summation, and the choice of the tetter / as 
the variable is arbitrary; that is, we could have used any other Setter, such as f or k. Or, 
in notation, 

. v„ ; v ( , ; . 

j’m i -m k~-m 

Here, the index of summation runs through all integers starting with its lower limit 
m and ending with its upper limit n. The uppercase Greek letter sigma, 2, is used to 
denote summation. We give some examples of summation notation. 

EXAMPLE 8 

Express the sum of the first 100 terms of the sequence {a„}, where a„ = 1 In for n 
1,2,3. 

Solution: The lower limit for the index of summation is 1, and the upper limit is 100. 
We write this sum as 

EDO , 

V 1 

p\ j 

■ 

EXAMPLE <) 

What is the value of Y ^ L 

Solution: We have 

y r • i : ■ 2 ; t > *4' - 
;j[ = 1+ 4 + 9+16 + 25 

= 55. m 

EXAMPLE 10 

What is the value of Y ,® 4 ( -- 1 )*? 

Solution: We have 

8 

V(-n* = • ir . ir-i :/• f l ; •• ( if 

t- 4 =. { +(-i) T i + (-i) + i 

- i. i 
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EXAMPLE U 


EXAMPLE 12 


Sometimes it is useful to shift the index of summation in a sum. This is often done 
when two sums need to be added but their indices of summation do not match. When 
shifting an index of summation, it is important to make the appropriate changes in the 
corresponding summand. This is illustrated by the following example. 


Suppose we have the sum 



j-l 


but want the index of summation to run between 0 and 4 rather than from l to 5. To do 
this, we let k = j-l. Then the new summation index runs from 0 to 4, and the term 
j 2 becomes (k + if. Hence 


5 4 

V(* + l) : 

jwi ir-0 


It is easily checked that both sums are 1 + 4 + 9 + 16 + 25 = 55. ■ 


A geometric progression is a sequence of the form 
a. ar, ar~, ar' . ar k , 

where a, the initial term, and r. the common ratio, are real numbers. Sums of terms 
of geometric progressions commonly arise; such sums are called geometric series. We 
w ill find a formula for S. the sum of the first n 4- I terms of a geometric progression 
with initial term a and common nonzero ratio r, that is, 

S = ^ ark 
i= n 

To compute S t first multiply both sides of the equality by r and then manipulate the 
resulting sum as follows; 

rS = r'y ar ! 

j - n 

= yv ,+i 

i-o 

n-H 

- V ar k 

jt= i 

= ^ or 1 + inr nJ 1 - a) (This equality is obtained by shifting the index of 
summation, setting k = j + 1.) 

- S + (ar ’ 1 ' f - a). 

From these equalities, we see that 

rS ~ !i + (flr" + l - a). 
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EXAMPLE 13 


EXAMPLE 14 


Solving for S shows that if r? I 

_ «^ +1 - « 
r- l“ 

If r = 1, then dearly the sum equals in -- 1 )a. I 


Double summations arise in many contexts (as in the analysis of nested loops in com¬ 
puter programs). An example of a double summation is 

4 3 

t-i j-i 

To evaluate the double sum, first expand the inner summation and then continue by- 
computing the outer summation: 

JVi ij = y^(i + 2 i + 3i) 

i=i j -1 /=i 

4 

(-1 

= 6+12 + 18 + 24 = 60. B 

We can also use summation notation to add all values of a function, or terms of an 
indexed set, where the index of summation runs over all values in a set. That is, we 
write 

V/(J) 

sES 

to represent the sum of the values fU), for all members s of 5. 


What is the value of 24j j? 

Solution: Since ? 4 x represents the sum of the values of s for all the members 
of the set {0,2,4), it follows that 

V x = o + 2 + 4 = 6. 

{ 0 , 1 , 4 } U 

Certain sums anse repeatedly throughout discrete mathematics. Having a collection 
of formulae for such sums can be useful, so Table 2 provides a small table of formulae 
for commonly occurring sums. 

We derived the first formula in this table in Example 12, The remaining three for¬ 
mulae give us the sum of the first n positive imergers, the sum of their squares, and 
the sum of their cubes. These three formulae can be derived in many different ways 
(for example, see Exercises 21 and 22 at the end uf this section), Also note that each 
formula, once known, can easily be proved using mathematical induction, the subject 
of Section 3,2. 

Example 15 illustrates how the formulae in Table 2 can be useful. 
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EXAMPLE 15 


EXAMPLE 16 


TABLE 2 Some Useful Summation 

Formulae. 


Sum j Closed Form 

V" ar k 

T - \ 


n{n + T) 

1 * 

~^r~ 

V" k 2 

n(n + 1)(2 n + 1) 


6 

ki 

—1.» i K 

„ ___ 

n 2 (n + 1 y 
' V 


■ 100 


Solution: First note that since X*™ = Xt-i* 2 + we have 

ign iro 49 

i ,t2 = X^"Z **• 

*-50 *=■ 1 *-l 

Using the formula XL i k 2 ~ n(n + 1 ){2n + 1 )/6 from Table 2, we see that 


100-101-201 49-50-99 „„„„ 

; -- 338,350 - 40,425 - 297,925. 


1 ^2 1IA/ ' 1U1 ■ £1/1 

6 


CARDINALITY (optional) 


Recall that in Section 1.4, the cardinality of a finite set was defined to be the number 
of elements in the set. It is possible to extend the concept of cardinality to all sets, both 
finite and infinite, with the following definition. 


DEFINITION 2. He sets A and B have the same cardinality if and only if there 
is a one-to-one correspondence from A to B. 


To see that this definition agrees with the previous definition of the cardinality of a finite 
set as the number of elements in that set, note that there is a one-to-one correspondence 
between any two finite sets with n elements, where n is a nonnegative integer. 

We will now split infinite sets into two groups, those with the same cardinality as 
the set of natural numbers and those with different cardinality. 

DEFINITION 3. A set that is either finite or has the same cardinality as the set of 
natural numbers is called countable. A set that is not countable is called tmeomuabk. 

We now 1 give examples of countable and uncountable sets. 


Show that the set of odd positi ve integers is a countable set. 

Solution: To show that the set of odd positive integers is countable, we will exhibit a 
one-to-one correspondence between this set and the set of natural numbers. Consider 


PDF created with pdfFactory trial version www.pdffactorv.com 













1.7 Sequences and Summations 7 7 


EXAMPLE 17 


[ 


4 5 6 


9 10 II 12 


1 ? 5 7 9 11 13 15 17 19 21 3 ■■ _ 

FIGURE 1 A One-to-One Correspondence Between N and the Set of Odd Positive 
Integers. 


the function 

f(n) = In - I 

from N to the set of odd positive integers. We show that f is a one-to-one correspondence 
by showing that it is kith one-to-one and onto. To see that it is one-to-one, suppose that 
fin) = f{m). Then 2n - 1 = 2m - 1, so that n = m. To see that it is onto, suppose 
that t is an odd positive integer. Then t is l less than an even integer 2k. where k is a 
natural number. Hence t = 2k - 1 = f(k). We display this one-to-one correspondence 
in Figure 1. ■ 


An infinite set is countable if and only if it is possible to list the elements of the set 
in a sequence (indexed by the natural numbers). The reason for this is that a one-to-one 
correspondence/ from the set of natural numbers to a set S can be expressed in terms. 

of a sequence a h a 2 . a„ .where = /(l), m - /(2),, ,, a„ = f(n) .For 

instance, the set of odd integers can be listed in a sequence a\, a 2) .... a,, __ w here 

a n = 2n - 1 . 

We now give an example of an uncountable set. 


Show that the set of real numbers is an uncountable set. 

Solution: To show that the set of real numbers is uncountable, we suppose that the 
set of real numbers is countable and arrive at a contradiction. Then, the subset of all 
real numbers that fall between 0 and 1 would also be countable (since any subset of 
a countable set is also countable; see Exercise 32 at the end of the section). Under 
this assumption, the real numbers between 0 and 1 can be listed in some order, say, 
f i, > 2 , r '},.... Let the decimal representation of these real numbers he 

n = O.diid 12^13^14 ■ ■ ■ 

r 2 = O-tfji ••, 

Li = 0-^31^32^33^34 - ■ ■ 
r4 = 0 , (G | ... 


where d.j E {0, 1,2.3,4, 5,6, 7, 8 . 9}. (For example, if rq - 0.23794102. ., we have 
= 2, (/12 = 3, d]^ - 7, and so on.) Then, form a new real number with decimal ex¬ 
pansion r - y).d\ diifof ■ ■, where the decimal digits are determined by the following 
rule: 



if d u = 4 
if da # 4. 
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(As an example., suppose that /q = 0.23794102..., r 2 = 0.44590138..., r } = 
0.09118764..., /"4 = 0.80553900..., and so on. Then we have r - O.c/jfAufoir ... = 
0.4544..., where d\ = 4 since u n ¥= 4, ch - 5 since d 21 ~ 4, d% =4 since rf 31 # 4, 
di - 4 since d+t = 4, and so on.) 

Ever)' real number has a unique decimal expansion (when the possibility that the 
expansion has a tail end that consists entirely of the digit 9 is excluded}. Then, the 
real number r is not equal to any of r 1: rj,.... since the decimal expansion of r differs 
from the decimal expansion of r l in the ith place to the right of the decimal point, for 
each i. 

Since there is a real number r between 0 and I that is not in the list, the assumption 
that all the real numbers between 0 and 1 could be listed must be false. Therefore, all the 
real numbers between 0 and 1 cannot be listed, so that the set of real numbers between 0 
and 1 is uncountable. Any set with an uncountable subset is uncountable {see Exercise 
35 at the end of this section). Hence, the set of real numbers is uncountable. ■ 


Exercises 

L Hind the following terms of the sequence fo, T } where 
= 2-(-31" t 5". 

a) tic b) ii, c) «4 d) y 5 

2. What is the term a* of the sequence \a n \ if a r _ equals 

a) 2" '? b. 7? 

c> i + i-ir? d> -(-2)«? 

3. What are the terms flu, ci, ui, and a-% of the sequence 
{a „}, where a„ equals 

a) 2" f 1? h) u- • n ;; ' 

C) [n/2j? d) [n/2j + fn/21? 

4. What ait the terms a u , a i, «i, and o 3 of the sequence 
\a„l where a h equals 

a) (-2f? b) 3? 

ci 7 + 4*? d) 2" h (-2)”? 

5. ' List the first 10 terms of each of the following se¬ 

quences . 

a) the sequence that begins with 2 ami in which each 
successive term is 3 more than the preceding term 

b) the sequence that lists each positive integer three 
times, in increasing order 

c) the sequence that lists the odd positive integers in 
increasing order, listing each odd integer twice 

d) the sequence whose nth term is n! — 2” 

e) Ih- sequence that begins with 3, where each suc¬ 
ceeding term is twice the preceding term 

f) the sequence whose first two terms are 1 and each 
succeeding term is the sum of the two preced¬ 
ing i cm is (This is the famous Fibonacci sequence, 
which we will study later in this text.) 

gi the sequence whose mh term is the number of bits 
in the binary expansion of the number n (defined in 
Section 2.3) 

h) tire sequence where (he nth term is the nuinher of 
letters in the English word for the index n 


6 , List the first 10 terms of each of the following se¬ 
quences. 

a) the sequence obtained by starting with If) and ob¬ 
taining each term by subtracting 3 from the previous 
term 

b) the sequence whose nth term is the sum of the first 
n positive integers 

c) the sequence whose mh term is 3 " - 2 " 

d) the sequence whose nth terra is [ v /nj 

e) the sequence whose first two terms are 1 and 2 and 
each succeeding term is the sum of the two previous 
terms 

fl the sequence whose nth term is the largest integer 
whose binary expansion (defined in Section 2.3) has 
n bits. (Write your answer in decimal notation.) 
gi the sequence whose terms are constructed sequen¬ 
tially as follows: start with 1 , then add !, then mul¬ 
tiply bv J. then add 2 , then multiply by 2 , and 
so on 

h) the sequence whose mh term is the largest integer 
k such that it! s n 

7. Find at least three different sequences beginning with 
the terms 1 , 2 ,4 whose tetins are generated by a simple 
formula or rule. 

8 . Find at least three different sequences beginning with 
the terms 3.5,7 whose terms are generated by a simple 
formula or rule. 

9. For each of the following lists of integers, provide a 
simple formula or rule that generates the terms of an 
integer sequence that begins with tile given list. 

a) 1,0, 1 , 1 . 0 . 0 , 1 , I, 1 . 0 , 0 , 0 , I,., 
b.) 1,2, 2, 5, 4, 4, 5, 6 , 6 , 7, 8 , 8 ,... 
c) (.0.2, 0,4, 0,8,0, 16,0,... 
di 3.6, 12, 24,48,96, 192,. . 


PDF created with pdfFactory trial version www.pdffactorv.com 





1.7 Exercises 79 


e) 15,S. 1, -6,-13,-20,-27,... 

0 3,5, 8,12,17,23,30,38,47,... 

g) 2, 16,54, 128,250,432,686,... 

h) 2.3, 7,25, 121,721,5041,40321,... 

10, For each of the following lists of integers, provide a 
simple formula or rule that generates the terms of an 
integer sequence that begins with the given list. 

a) 3.6, 11.18,27.38,51.66,83, 102,... 

b) 7, 11, 15, 19,23, 27,31,35,39,43,... 

c) l. io, ii, loo. ioi, no, in, looo, looi, 1010, 
1011 ,... 

d) 1,2,2, 2, 3, 3, 3, 3, 3, 5, 5, 5, 5, 5,5.5,... 

e) 0, 2, 8 , 26, 80, 242, 728, 2186, 6560, 19682,.,. 

f) I. 3, 15, 105, 945, 10395, 135135, 2027025, 
34459425,.,. 

g) 1 , 0 , 0 , 1 , 1 , 1 , 0 , 0 , 0 , 0 , 1 , 1 , 1 , 1 , 1 ,... 

b) 2, 4, 16.256, 65536,4294967296,.,, 

*H. Show that if <i a denotes the nth positive integer that is 
not a perfect square, then a n = n + { v /n}, where {,t} 
denotes the integer closest to the real number t, 

*12. Let be the nth term of the sequence 1,2, 2, 3,3,3, 
4, 4, 4, 4, 5, 5, 5, 5, 5, 6 , 6 , 6 , 6 , 6 , 6 ,..constructed 
by including the integer k exactly k times. Show that 

ft, = U y '2n + |J- 

13. What are Lhe values of the following sums? 

5 4 

a) V(A+ 1 ) b) T(- 2 )f 


c) V 3 d) X;2- -i - 2 ; 

I .1-0 

14. What are the values of the following sums, where S - 
(1.3, 5,7)? 


a) X j 

j-CS 


b) v 


G) Mu /j) d> v 1 

JE.V jes 

15. W'hat is the value of each of the following sums of terms 
of a geometric progression -1 

a) X 3 ■ V b) V V 


-o 


t-i 


c) X(-3)' dl T 2 1 (— 3)? 
j - ha 

16. Find the value of each of the following sums. 

s s 

a) XU +(-])/) bl V(3 J - 2 J ) 


c) XU - 3 y + 3-2U d) X(2 J+1 -V) 

;M! J-B 

17. Compute each of the following double sums. 


a) X X (i + j) 

i /-« 
c) X X i 

; - i ; (t 


b) V X ( 2 i + 3/) 

i= 0/=0 

d) T X ij 

t-Qi-1 


18. Compute each of the following double sums, 
a) X X (i - J ) b) X X (3t + 2 J) 

i*l j = l M j-V 


c) x x j d) X x o 3 

i=l/ = 0 1=0/=0 

19. Show that X"=i( a ; ~ flj-i) = a B - (Jo where 
an, fli, is a sequence of real numbers. This type 
of sum is called telescoping. 

20. Use die identity 1/1 k(k + 1)) - 1 Jk - 1/Ur + 1) and 
Exercise 19 to compute XXi lKKk + U)- 

21. Sum both sides of the identity k : - (k - \ f = 2k - 1 
from k = 1 to it = n and use Exercise 19 to find 

aj a formula for X* = i - 1 ) (the sum of the first n 

odd natural numbers), 
b) a formula for XX i k. 

*22, Use the technique given in Exercise 19, together 
with the result of Exercise 13b, to find a formula for 

v” a 

23. Find Xi“ 1(M U (Use Table 2.) 

24. Find (Use Table 2.) 

*25, Find a formula for XXoiX^j, when m is a positive in¬ 
teger (Him: Use the formula for X i k 1 .) 

*26. Find a formula for XXol vfcj, when m is a positive 
integer. {Hint: Use the formula tor XX l^ 1 ) 

There is also a special notation for products. The product of 
a„, ...a„ is represented by 


fix- 


27, What are the values of the following products? 

a) n,“=o' b) n*. s i 

«) n^t-iy din, 1 !^ 

The value of the factorial function at a positive integer n, 
denoted by nl. is the product of the positive integers from 1 
to n, inclusive. Also, we specify that 0! = 1. 


28. Express n\ using product notation. 

29 Fii.J ,!. 

30. Find n%o jl 

31. Determine whether each of the following sets is count¬ 
able or uncountable. For those that are countable, ex¬ 
hibit a one-to-one correspondence between the set of 
natural numbers and that set, 

a) the negative integers 

b) the even integers 

e) fee real numbers between 0 and f 

d) integers that are multiples of 7 
*32. Determine whether each of the following sets is count¬ 
able or uncountable. For those that are countable, ex¬ 
hibit a one-to-one correspondence between the set of 
natural numbers and that set, 

a) integers not divisible by 3 

b) integers divisible by 5 but not by 7 
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c) the real numhers with decimal representathms con¬ 
sisting of all is 

di the real numbers with decimal representations of all 

1 s or 9s 

33. If A is an uncountable set anti B is a rxiumable set, must 
A - B be uncountable' 1 

34, Show that a subset of a countable set is also count¬ 
able. 

35. Shosv that if .4 is an uncountable set and 4 C 5, then 
B is uncountable. 

36, Show that the union of two countable sets is countable. 
'*37. Show that the union of a countable number of countable 

sets is countable. 

*38, A real number is called rational if it can be written as 
the quotient of two integers. Show that the set of ratio¬ 
nal numbers between 0 and 1 is countable, {Hint: List 
the elements of this set in order of increasing p 4 - q, 
w here ft is the numerator anti q is the denominator of a 
fraction p/q in lowest terms.) 


*39. Show that the set of all bit strings is countable. 

*40. Show that the set of real numbers that are solutions of 
quadratic equations ax 2 + bx + c = 0, where a, b, and 
r are integers, is countable. 

*41. Show that the set of all computer programs in a par¬ 
ticular programming language is countable. (Hint: A 
computer program written in a programming language 
can be thought of as a string of symbols from a finite 
alphabet.) 

*42. Show that the set of functions from the positi ve integers 
to the set {0. 1,2,3,4,5,6,7,8,9} is uncountable. (Him. 
First set up a one-to-one correspondence between the 
set of real numbers between 0 and 1 and a subset of 
these functions. Do this by associating to the real num¬ 
ber OJidi the function/ with fin) = d„.) 

*43. We say that a function is computable if there is a com¬ 
puter program that finds the values of this function. Use 
Exercises 4! and 42 to show that there are functions 
that are not computable. 


L8_ 

The Growth of Functions 

INTRODUCTION 


Suppose that a computer program reorders any list of n integers into a list where the 
integers are in increasing order. One important consideration concerning the practi¬ 
cality of this program is how long a computer takes to solve this problem. An anal¬ 
ysis may show that the time used to reorder a list of n integers (where these inte¬ 
gers are less than some specified size) is less than f(ri) microseconds, where f(n) = 

1 00n log n + 25/i + 9. To analyze the practicality of the program, we need to under¬ 
stand how quickly this function grows as n grows. This section reviews some important 
methods used in estimating the growth of functions. We will introduce the notation 
most commonly used in the analysis of the growth of functions, namely, b[g-0 no¬ 
tation. We will develop some useful results about the growth of functions using tnis 
notation. 


BIG-0 NOTATION 


The growth of functions is often described using a special notation. The following def¬ 
inition describes this notation. 


DEFINITION 1. Let/ and g be functions from the set of integers or the set of real 
numbers to the set of real numbers. We say that f(x) is OffffxViifthkrr 
C and k such that 

I/O)! * CjgfOt 

whenever x > k. (This is read as "/{ x) is big-ofa of g(x)”) 
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EXAMPLE 1 


Remark: To show fix) is 0(g(x)), we need only find one pair of constants C and k such 
that |/(Jf)| ^ C\g{x)\ if x > k. However, a pair C, k that satisfies the definition is never 
unique. Moreover, if one such pair exists, there are infinitely many such pairs. A simple 
way to see this is to note that if C, k is one such pair, any pair C, k' with C < C' 
and k < k‘ also satisfies the definition, since j/(x)| < C|g(je)| C'\g(x)\ whenever 
x > k' > k. 


Show that f(x') ~ a 2 + 2.t + l is 0(x 2 ), 

Solution: Since 

0 == x 1 + 2x + 1 < x 2 + 2x 2 + x 2 = 4.x 2 

whenever x > 1, it follows that fix) is 0(x 2 ), (To apply the definition of big-O notation 
here, take C - A and k - 1. It is not necessary to use absolute values here since all 
functions in these equalities are positive when .t is positive.) 

Another approach is to note that when x > 2, it follows that 2,r ^ x 1 , Conse¬ 
quently, if x > 2, we see that 

0 ’ -r + 2x + 1 s ..v 2 + x 2 + x 2 = 3x z . 

(We apply the definition with C = 3 and k - 2.) 

Observe that, in the relationship fix) is Oix 1 ), x 2 can be replaced by any function 
with larger values than x 2 , for example, fix) is 0{ Cl fix) is Oix 2 f 2x + 7), and so 
on. It is also true that x 2 is Ofx 1 + 2x + 1), since x 2 < x 2 + 2.r + 1 whenever i s l. 
Figure 1 illustrates that x 2 + 2.x + 1 is 0(x 1 ). ■ 


Note that in Example l we have tw'o functions, fix) = x 2 + 2x + 1 and gix) - 
x 2 , such that fix) is 0(g(jc)) and gut is 0{f(x)) —the latter fact following from the 
inequality x~ ■& x 2 + 2x + [, which holds for all nonnegative real numbers x. We say 
that two functions fix) and gix) that satisfy both of these big-0 relationships are of the 
same order. (See pages 88—90 ) 



FIGURE 1 The Function r 2 + 2r + 1 Is Oix 2 ). 
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Remark: The fact that fix) is 0(#(x)) is sometimes written f(x) = 0Q>( *)), How¬ 
ever, the equals sign in this notation does not represent a genuine equality. Rather, this 
notation tells us that an inequality holds relating the values of the functions/ and g for 
sufficiently large numbers in the domains of these functions. 

Big-0 notation has been used in mathematics for almost a century. In computer 
science it is widely used in the analysis of algorithms, as will be seen in Chapter 2. 
The German mathematician Paul Baehmann first introduced big-0 notation in 1892 in 
an important book on number theory. The big-0 symbol is sometimes called a Lan¬ 
dau symbol after the German mathematician Edmund Landau, who used this notation 
throughout his work. The use of big-0 notation in computer science was popularized 
by Donald Knuth, who also introduced the big-Ornega and big-Theta notations defined 
later in this section. 

When fix) is O(gix)), and h(x) is a function that has larger absolute values than 
gix) does for sufficiently large values ofx, it follows that fix) is 0(h(x)). In other words, 
the function g{x) in the relationship f\x) is 0(g(x)) can be replaced by a function with 
larger absolute values. To see this, note that if 

I/Mi *' C|g(4j if a > k, 
and if jfc(x)| > \g{x)l for all a > k, then 

l/(x)l - C\h(x) ifx> k. 

Hence, fix) is 0(/i(x)) 

When big-0 notation is used, the function g in the relationship f{x) is 0(g(x)} is 
chosen to be as small as possible (sometimes from a set of reference functions, such as 
functions of the form a - ”, where n is a positive integer). 

In subsequent discussions, we will almost always deal with functions that take on 
only positive values. All references to absolute values can be dropped when 'working 
with big-0 estimates for such functions. Figure 2 illustrates the relationship f{x) is 
0(g(x)). 

The following example illustrates how big-0 notation is used to estimate the growth 
of functions. 


Paul Gustav Heinrich Baehmann (1837-19201. Paul Baehmann, the son of a Lutheran pastor, shared his 
yfgf} father’s pious lifestyle and love of music. His mathematical talent was discovered by one of liis teachers, 
even though he had difficulties with some of his early mathematical studies. After recuperating from tu¬ 
berculosis in .Switzerland, Baehmann studied mathematics, first at the University of Berlin and later at 
Gottingen, where he attended lectures presented by the famous number theorist Dirichlet. He received 
his doctorate under the German number theorist Rummer in 1862; his thesis was on group theory. Barh- 
mann was a professor at Breslau and later at Miinsfer. After he retired from his professorship, he con¬ 
tinued his matheirtaiic.il writing, played the piano, and served as a music critic for newspapers. Bach- 
mann's mathematical writings include a five-volume survey of results and methods in number theory, a 
two-volume work on elementary number theory, a book on irrational numbers, and a hook on the famous 
conjecture known as Fermat’s Last Theorem. He introduced big-f7 notation in his 1892 book Anafyiiicfie 
Zahtenthtirrie. 

Ldmund Landau 11877-19381. Fdmurtd Landau, the son of a Berlin gynecologist attended high school 
ty$lf and university in Berlin, He received his doctorate in 1899, under the direction of frobemus. Landau first 
taught at the University of Beilin and then moved to Gottingen, where he was a full professor until the 
Nazis forced him to stop teaching. Landau’s main contributions to mathematics were in die field of analytic 
number theory. In particular. He established several important results concerning the distribution of primes. 
He authored a three-volume exposition on number theory as well as other books on number theory and 
mathematical analysis. 
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FIGURE 2 The Function f(x) Is Qigixl). 



Donald E. Knuth (horn 1938). Knuth grew up in Milwaukee, where his father taught bookkeeping at a 
Lutheran high school and owned a small printing business. He was an excellent student, setting academic 
achievement awards. He applied his intelligence in unconventional ways, winning a contest when he was 
in the eighth grade by finding over 4,500 words that could be formed from the letters in “Ziegler’s Gian! 
Bar.” This won a television set for his school and a candy bar for everyone in his class. 

Knuth had a difficult time choosing physics over music as his major at the Case Institute pf'Technology. 
He then switched from physics to mathematics, and in 1960 he received his bachelor of science degree, 
simultaneously receiving a master of science degree by a special award of the faculty who considered his 
work outstanding. At Case, he managed the basketball team and applied his talents by constructing a formula 
for the value of each player This novel approach was covered by Newsweek and by Walter Cronkite on the 
CBS television network. Knuth began graduate work at the California Institute of Technology in i960 and 
received his Ph.D. there in 1963. During this time he worked as a consultant, writing compilers for different 
computers, 

Knuth joined the staff of the California Institute of Technology in 1963, where he remained until 1968, 
when he took a job as a full professor at Stanford University, He retired as Professor Emeritus in 1992 to 
concentrate on writing. He is especially interested in updating and completing new 1 volumes of his series The 
An of Computer Programming, a work that has had a profound influence on the development of computer 
science, which he began writing as a graduate student in 1962, focusing on compilers. In common jargon, 
“Knuth,” referring to The Art of Computer Programming, ha? come to mean the reference that answers all 
questions about such topics as data structures and algorithms 

Knuth is the founder of the modem study of computational complexity. He has made fundamental 
contributions to the subject of compilers His dissatisfaction with mathematics typography sparked him tu 
invent the now widely used TeX and Merer:-:’ systems. TeX has become a standard language for computer 
typography. Two of the many awards Knuth has received are the 1974 Turing Award and the 1979 National 
Medal of Technology, awarded to him by President Carter. 

Knuth has written for a wide range of professional journals in computer science and in mathematics 
However, his first publication, in 1967, when he was a college freshman, was a parody of the metric system 
called “The Potrwbie Systems of Weights and Measures.” which appeared in MAD Magazine and has been 
in reprint several times. He is a church organist, as his father was. He is also a composer of music for the 
organ. Knuth believes that writing computer programs can be an aesthetic experience, much like writing 
poetry or composing music. 

Knuth pays $2.56 for the first person to find each emir in his books and $0,32 for significant sugges¬ 
tions. If you send him a letter with an error ( you will need to use regular mail, since he has given up reading 
e-mail), he will eventually inform you whether you were the first person to tell him about this error Be 
prepared for a long wail, since he receives an overwhelming amount of mail. (The author received a letter 
years after sending an error report to Knuth, noting that this report arrived several months after the firs! 
report of this error,} 
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EXAMPLE l 


EXAMPLE 3 


THEOREM 1 


EXAMPLE 4 


Show [hat 7x 2 is 0{x i ). 

Solution: The inequality lx 2 < ,r holds whenever x > 7. (We see this by dividing 
boih sides of this inequality by x 2 .) Hence, lx 2 is O(.x-), ticking C = ] and A = 7 in 
the definition of big-0 notation. g 


Example 2 shows that 7r is <9(.r 3 ). Is it also true that r } is 0(7x J )7 


Solution: 1b determine whether .r 3 is (Mix 2 ), it is necessary to determine whether 
there arc constants C and A' such that x 3 < C(lx 1 ) whenever c > A. This inequality is 
equivalent to the inequality x < 1C, which is obtained by dividing both sides by r. 
No such 0 can exist since x can he made arbitrarily large. Hence x* is not (Mix 1 ). ■ 

Polynomials can often be used to estimate the growth of functions. Instead of ana¬ 
lyzing the growth of polynomials each time they occur, we would like a result that can 
always be used to estimate the growth of a polynomial. The following theorem does 
this. It shows that the leading term of a polynomial dominates its growth by asserting 
that a polynomial of degree n or less is 0(x /! ), 


Let f{x) = anX* + Qz-ix* 1 f + * ■ • + dj x + ao t where <j<j, aj,,.., a n are real 
numbers, Then / (x) is Oixf 1 )- 

Proof: Using the triangle inequality, if jc > 1 we have 

= \a,x" + a K -»-■■■+-«,, t - y flg | 

— + jttn-i|x N 1 + ■ ■* - 1 - ]a||.r + |u ( )j 

= x" (!«„{ + |a ff _ [|ix + ■ ■ ■ -r jfli|/.t' f 1 + luoj/.v' 7 j 
< .^(JaJ + lfl, ii+ •■■+(a,| + ;u 0 |). 

This shows that 

I/Ml ' Cx ” 

where C = |(V - + ■ ■ - 4 |fitpl whenever x > 1. Hence, f(x) is 0(. t' 3 ). Q 

We now give some examples involving functions that have the set of positive inte¬ 
gers as their domains. 


How can big-0 notation be used lo estimate the sum of the first n positive integers 1 ' 

Solution: Since each of the integers in the sum of the first n positive integers does not 
exceed n, it follows that 

1+2 F ■ ■ ■ + n < n + n + • ■ ■ 4 n - tf. 

From this inequality it follows that 1 + 2 + 3 ^ ■ ■ - + n is Off ), taking C - 1 and 
A - 1 in the definition of big-0 notation. (In this example the domains of the functions 
in the big-0 relationship arc the set of positive integers, ! g 
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EXAMPLE 5 


EXAMPLE (> 


In the next example big-0 estimates will be developed for the factorial function 
and its logarithm. These estimates will be important in the analysis of the number of 
steps used in sorting procedures. 


Give big-<? estimates for the factorial function and the logarithm of the factorial func¬ 
tion, where the factorial function f(n) - n! is defined by 

n! = 1-2-3. n 

whenever n is a positive integer, and tl! = 1. For example, 

1! = 1, 21 = 1-2 = 2, 3! = 1-2-3 = 6, 4! = 1 - 2 ■ 3 ■ 4 = 24. 

Note that the function «! grows rapidly. For instance, 

20! - 2,432,902,008,176,640,000. 

Solution: A big-0 estimate for n! can be obtained by noting that each term in the prod¬ 
uct does not exceed n. Hence, 

n! = 1-2-3. n 

s n ■ n ■ n . n 

= n\ 

This inequality shows that n\ is O(tf). Taking logarithms of both sides of the inequality 
established for«!, we obtain 

log n < log «" = n log n. 

This implies that log «! is £?{nlogn). p 


In Section 3.2 wc will show that 
n < 2" 

whenever n is a positive integer. Using this inequality we can conclude that n is 0{2 n ). 
(Take k = C = 1 in the definition of big-Cf notation.) Since the logarithm function is 
increasing, taking logarithms (base 2) of both sides of this inequality shows that 

log n < n. 

It follows that 

log n is O(n). 

(Again we take C = k - 1 in the definition of big-0 notation.) 

If we have logarithms to a base b, where b is different from 2, we still have log fe n 
is 0{n) since 


, log n n 

logs « = < - 

log b log b 


whenever n is a positive integer, (We have used Theorem 3 in Appendix 1 to see that 
logjfl = log n/iog h). g 
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THE GROWTH OF COMBINATIONS OF FUNCTIONS 

Many algorithms are made up of two or more separate subprocedures. The number 
of steps used by a computer to solve a problem with input of a specified size using 
such an algorithm is the sum of the number of steps used by these subprocedures. To 
give a big-0 estimate for the number of steps needed, it is necessary to find big-<9 
estimates for the number of steps used by each subprocedure and then combine these 
estimates, 

Big-f9 estimates of combinations of functions can be provided if care is taken when 
different big-0 estimates are combined. In particular, it is often necessary to estimate 
the growth of the sum and the product of two functions. What can be said if big 69 
estimates for each of two functions are known? To see what sort of estimates hold for 
the sum and the product of two functions, suppose that f\ix) is O(gff jc)) and fjix) is 
0(g 2 (x)). 

From the definition of big-0 notation, there are constants C\, €%, k\ , and k-, such 
that 

1/iWi - GbtMl 
when x > k\, and 

f'J.x ' - C 2 IT 2 O) 

when ? > k- 2 - To estimate the sum of ft ( r) and note that 

lt/i + / 2 X-OI - !/j(.r) + f 2 {xt\ 

— l/i(-r): + LftCr) (using triangle inequality \a + b\ < jo| + hi). 

When x is greater than both k\ and C, it follows from the inequalities for /](.r)i and 
that 

l/iU)l <C 1 |i’iW| + C 2 |g 2 (x)| 

- Cijg(A)| + c 2 \m\ 

= (Cl + c ~ 1 Vi rib 

= C\g{x)\, 

where C = C^C 2 andgix) ~ max(]gt (x)|,[^a(. x}|). (Here max(a, b) denotes the max¬ 
imum, or larger, of a and b.) 

This inequality shows that j(/i + / 2 }(x)[ < C|g(y) whenever r > k, where k - 
rnaxi.ft. k 2 ). We state this useful result as the following theorem. 

THEOREM 1 

Suppose that f t (x) is OigAx)) and f 2 (x) is Oig 2 (x)). Then C/i + M*) fc 
0(ma(g } (x), g 2 (x))). 

COROLLARY i 

We often have big-0 estimates for ft and f 2 in terms of the same function g. In 
this situation, Theorem 2 can be used to show that (/, + f z )(x) is also 0(g(x)l since 
max(£(x),g(x» = g{x). This result is stated in the following corollary. 

Suppose that f{x) and f 2 (x) are both 0(g(x)). Then (/, 4 />)(*) Is CHg{x)). 
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In a similar way big-0 estimates can be derived for the product of the functions f\ 
and /i. When x is greater than max(G, k 2 ) it follows that 

K/i/a)Wi = |/i(x)||/ 2 (x)| 

• • 0 g tOsr, 

- CjC 2 |( ?lft X*)| 

£ G(«gig 2 ){x)[, 

where C = QCi, From this inequality, it follows that /,( x)h\x) is 0(g jg 2 ), since 
there are constants C and k— namely, C = CiC 2 mdk = max(A), since j(/i / 2 )(jc)) 

£ C gi(x)^ 2 (4f whenever _r > k. This result is stated in the following theorem. 

THEOREM 3 

Suppose that /i (x) is 0C?, for)) and f 2 (x) is CH^ 2 (x». Then ifiM*) 

The goal in using big-0 notation to estimate functions is to choose a function g( (x) 
that grows relatively slowly .so that fix) is 0(g(x)). The following examples illustrate 
how to use Theorems 2 and 3 to do this. The type of analysis given in these examples is 
often used in the analysis of the time used to solve problems using computer programs. 

EXAMPLE 7 

Give a big-0 estimate for f(n) ^ 3/ilog(/ri) + (tr + 3)logn, where n is a positive 
integer. 

Solution: First, the product login!) will be estimated. From Example 5 we know 
that login!) is Oin log n). Using this estimate and the fact that 3 n is Oin), Theorem 3 
gives the estimate that 3 b login!) is 0(n 2 log n). 

Next, the product (n 2 4- 3) log n will be estimated. Since (n 1 4 3) < 2 n 2 when 
n > 2, it follows that n 2 -f 3 is Oin 7 '), Thus, from Theorem 3 it follows that (/r + 3) log n 
is Oi/r log n). Using Theorem 2 to combine the two big-0 estimates for the products 
shows that f(n) =-■ 3/J login!) F ar log n is Oin 1 logn). ■ 

EXAMPLE 8 

Give a big-0 estimate for fix) = \x + 1) logf.v 2 + 1) t 3x 2 . 

Solution: First, a big-0 estimate for {x + 1) logt.r + 1) will be found. Note that {x r 1) 
is Of.vj. Furthermore, ,r f 1 < 2,r 2 when x > I. Hence, 

log( .U 4 1) si log(2,r > = log 2 4- log.r = log 2 + 21og,r 2 3 log ,v. 

if x > 2. This shows that logi c 2 t 11 is 0(log x). 

From 'I’heorem 3 it follows that (x 4 1) logt.r 4-1) is 0(x log x). Since 3,r is 0(.r}, 
Theorem 2 tells us that f{x) is 0(tnax(.rlog x, x 1 )), Since x logx s j-, for x > |, it 
follows that f\\) is 0u\. • 


As mentioned before, big-0 notation is used to estimate the number of operations 
needed to solve a problem using a specified procedure or algorithm. The f unctions used 
in these estimates often include the following: 

1, Iflgn, n. n log n, n 2 , 2", r.l 
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FIGURE 3 A Display (if the Growth of Functions Commonly Used in Big-0 Estimates, 

Using calculus it can be shown that each function in the list is smaller than the suc¬ 
ceeding function, in the sense that the ratio of a function and the succeeding function 
tends to zero as n grows without bound. Figure 3 displays the graphs of these functions, 
using a scale for the values of the functions that doubles for each successive marking 
on the graph. 

BIG-OMEGA AND BIG-THETA NOTATION 


Big-0 notation is used extensively to describe the growth of functions, but it has limi¬ 
tations. In particular, when f(x) is 0(g(x )}, we have an upper bound, in terms of g(x), 
for the size of /(„t) for large values of x. However, big-0 notation does not provide 
a lower bound for the size of fix) for large x. For this, we use big-Omega notation. 
When we want to give both an upper and a lower bound on the size of a function /(*). 
relative to a reference function gut, we use big-Theta notation. Both big-Omega and 
big-Tlieta notation were introduced by Donald Knuth in the 1970s. His motivation for 
introducing these notations was the common misuse of big-0 notation when both an 
upper and a lower bound on the size of a function are needed. 

We now define big-Omega notation and illustrate its use. After doing so. wc will 
do the same for big- Theta notation. 

There is a strong connection between big-0 and big-Omega notation. In particular. 
f{x) is il(glx')) if and only if g(x) is O(fix)). We leave the verification of this fact as 
a straightforward exercise for the reader. 


DEFINITION 2. Let / and g be functions from die set of integers or the set of real 
numbers to the set of real numbers. We say that fix) is £l(g{x)) if (here are positive 
constants C and k such that 

i/wi ctoi 

whenever x > k. (This is read as u f{x) is big-Omega of g(x)”). 
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EXAMPLE 9 


FX AMPLE 1(1 


The function f{x) = 8.x 3 + 5x 2 + 7 is fl(g(x)), where #(x) is the function g{x) * x\ 
This is easy to see since fix) - 8.Y + 5.v : + 7 > 8x 3 for all positive real numbers x. 
This is equivalent to saying that j?(x) = x 3 is 0(8r ! + 5x 2 +7), which can be established 
directly by turning the inequality around. ■ 


Often, it is important to know the order of growth of a function in terms of some 
relatively simple reference function such as x" when n is a positive integer or c r , where 
c > 1. Knowing the order of growth requires that we have both an upper bound and 
a lower bound for the size of the function. That is, given a function fix), we want 
a reference function g{x) such that fix) is 0(g(xl) and fix) is fl(gfr)). Big-Theta 
notation, defined as follows, is used to express both of these relationships, providing 
both an upper and a lower bound on the size of a function. 

DEFINITION 3, Let / and g be functions from the set of integers or the set of real 
numbers to the set of real numbers. We say (hat fix) is 0(g(x)) if f(x) is CKg(x)) 
and f(x) is ft (git)). When fix) is %ig(x}\ we say (hat “/ is big-Ttaeta of g(x)" 
and we also say that fix) is of order y(i). 


When /( x) is E%(.v)), it is also the case that g(.r) is @(/(j r)). Usually, when big- 
Theta notation is used, the function gix) in 0(g(x)) is a relatively simple reference 
function., such as x n , c 1 , log x, and so on, while f(x) can be relatively complicated. 


We showed (in Example 4) that the sum of the first n positive integers is 0(n 2 ). Is this 
sum of order « 2 ? 

Solution: Let fin) - 1 + 2 + 3 + —tn. Since we already know thal / (n) - Oitf), to 
show that / in) is of order rr we need to find a positive constant C. such that f(n) > Cn 2 
for sufficiently large integers n. To obtain a lower bound for this sum, we can ignore 
the first half of the terms. Summing only the terms greater than \ni2\ we find that 

1 -r 2 + ■ ■ ■ + n 5; [n/2] + ([n/2] + 1) -i- ■ ■ ■ 4 n 

£ fn/21 + fn/21 + ■ ■ ■ + \nfl\ 

- (« - [n/2] t 1) fn/21 
> (b/2)(h/2) 

- n 2 /4. 

This shows that fin) is ft(n 2 ), We conclude that f(n) is of order rr, or in symbols, 
fin) is <“>(>r). | 


We can show thal fix} is >< if wo can find positive real numbers C| and Cf 
and a positive real number k such that 

EYsMl * |/(x)| s G|g(t->| 

whenever x £ k. This shows that fix) is ()■ g(x)) and fix) is fl(,?(.r)). 
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EXAMPLE II Show that 3a 2 f 8* log* is 0{a 2 ). 

So luii on: SinceO < 8 a log x £ 8x 2 , it follows that 3 x 1 4 8x log a s 11 jc 2 for jc > 1. 
Consequently, 3.r + 8x log x is 0(x 2 ). Clearly, x l is 0(3 x 2 + 8x log x). Consequently, 
3 a 2 -t- 8 a log x is (-)< r). ■ 


One useful fact is that the leading term of a polynomial determines its order. For 
example, if /(*) = 3.C + x 4 + 17.r' + 2, then fix) is of order a 5 . This is stated in the 
following theorem, whose proof is left as an exercise at the end of this section. 

THEOREM 4 Let fix) = a**" 4 _ ] x*~ 1 + - -h a [ x 4or >( where % aj, ..., a* are real numbers 

with a K 7 s 0. Then f(x) is of order x 


EXAMPLE 12 


The polynomials 3a 8 •+ IO C +221 a 2 h 1444, x 19 — 18 a 4 - 10112, and ~a w +40001 A 93 
■+ 100003* are of orders x s , a 19 , and a", respectively. ■ 


Unfortunately, as Knuth observed, big-0 notation ts often used by careless writers 
and speakers as if it had the same meaning as big-Tbeta notation. Keep this in mind 
when you sec big-0 notation used. The recent trend has been to use big-Theta notation 
whenever both upper and lower bounds on the size of a function are needed. 


Exercises 

1 Determine whether each of the following functions is 
0 ( 0 . 

at /td - Hi h) ft x) ■= 3 i 4 7 

el /(.si C • v • 1 di /(,v> = f log x 

e) ft .0 l aJ f) f{x) - \xW 

2. Determine whether each of the fol lowing functions is 
Ol-C). 

at fix) -- I7.v f II b) fix) -= r + 1000 

el /ui - a log v d) fix) = .0/2 

et /(<) ■■ ft fix) = [aM.C 

5. Use (he definition of the fact that fix) is f%(x)) to 
show that .c 4 + 9x 3 4 4a 4 7 is D(x 4 }. 

4. Use the definition of the fact that fix) is Uigf.v)) to 
show that 2' 4 17 is Gh-Vj- 

5. Show that i t - 1 IVt.v-r !) is 0 (a). 

6. Show that {x’ r 2.v)/(2x I 11 is Of.r* 1 ). 

7. hind the least integer n such that /(.a) is 0 ( a") for each 
of the I til lowing functions. 

a} /(.vi ■ ■ 2 a 2 4 .r log a 
h.) fix) - 3a’ t- (log a-) 4 

c) /(v.> i.r + .T -r i)./ x ‘ < I ) 
tl) fix i - I.r 4 v 5 log x.i/(.r 4 t- I) 


8. hind the least i nteger n such that /(a) is Q{x n ) for each 
of the follow ing functions, 

a) f(x) - 2x 2 4 .v’ log a 
b} fix) = ?.C 4 (log x) 4 

C} fix) =: (A 4 + A 2 -r I )/{A 4 + 1 ) 

d) fix) = U- 1 4 5 log v)/(A 4 + 1) 

9, Show that x‘ + 4x4 17 i s 0(x 5 ) but that a 3 is not Off 1 + 
4a 4 17). 

10. Show that v 3 is 0(.f ) but that a 4 is not O(.U). 

11. Show that 3 a 4 4 I is U(x 4 /2) and U/2 is 0(3x 4 4 I), 

12. Show that v log x is 0{x ! ) but that x 2 is not Of'x log a) 

13. Show that 2" is 0(3") bus that 3“ is not 0(2"). 

14. Is it true that x 3 is Oir.v.ii. if g is the given func¬ 
tion? [For example, g: v) = x + 1, this question asks 
whether a 2 is 0{x + I).] 

a) g(x) = x 1 

b) g( n - a 3 

d v i) - a 2 a- a 3 
(t) g(x) = A 2 - A 4 

e) g(x) - y 

f) six) - U/2 

15. Explain what it means for a function to be Of I j 

16. Show that if fix) is 0(a). then f(x) is Of r). 
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17. Suppose that f(x). g(x). and h(x) are functions such 
that /Uj is 0(g{x)) and gu) is O(hix))- Show that 
fix) is 0(h(x)). 

18. Let k be a positive integer. Show that l i +2*4--- + jt t 
is 0{n t+l ). 

19. Give as good a big-<9 estimate as possible for each of 
the following functions, 

a) [n 2 4- 8)(u + 1) 

b) in log n 4 n-'iin 1 ' + 2) 

c) (it! + 2"J(n s 4 Iog(rr 4 1)) 

20. Give a big-f? estimate for each of the following func¬ 
tions, For the function g in your estimate fix) is 0(g), 
use a simple function g of smallest order. 

a) (n 1 + n 2 logn)(log n 4 1) (17 log a + 19)(tr -r 2) 

b) (2 n + /r»(V 4 3") 

c) (n n + n2" 4 + 5’ r ) 

21. Give a big-0 estimate for each of the following func¬ 
tions. For the function g in your estimate that fix) is 
Oi f’( j )) use a simple function j? of the smallest order. 

a) n log(n 3 + 1) 4 re log n 

b) (nlog n 4 I) 3 + (log n j- l)(jr + )) 
e) n 2 ’ 4 n"~ 

22. For each function in Exercise 1, determine whether that 
function is 11(4) and whether it is 0(x), 

23. For each function in Exercise 2, determine whether that 
function is LH.r ) and whether it is 0(,r), 

24. a) Show that lr 4 7 is 0(c). 

b) Show that 2x 3 t x - 7 is 0(,r 2 ). 

c) Show that [x 4 ]/2J is H(.v). 

d) Show that logic 3 + I) is 0(kig 2 x), 

e) Show that log [(] x is 0(log 2 ,r). 

25. Show that fix) is ®(g(x}) if and only if /{ x) is C%(xJ) 
and g(x) is Oif(x)). 

26. Show' that if f(x) and g(x) are functions from the set 
of real numbers to the set of real numbers, then f(x ) is 
O(gix)) if and only if six} is Gi/ttjt. 

27. Show that if fix) and g(x) are functions from the set 
of real numbers to the set of real numbers, then fix) 
is Wig ri) if and only if there are positive constants 
k, Ci, and C 2 such that C||g(jr)f £ |/U)[ s CzLgUi! 
whenever x > k. 

28. a) Show that 3;r 4- x + 1 is 0(3.v 2 ) by directly finding 

the constants k.C i, C, in Exercise 27. 
b) Express the relationship in part (a) using a picture 
showing the functions 3.r 2 4 x + 1, Ci ■ 3a 3 , and 
Ci 3x\ and the constant k on the t-axis. where 
Ci, C;, and k are the constants you found in part (a) 
to show that 3a- 2 4 .r 4 1 is 0(3x 2 ). 

29. Express the relationship fix) is W(g(c)) using a 
picture. Show the graphs of the functions fix), 

C. ?n'. and Cj]g(xl]. as well as the constant k on the 
a-ax is. 

30. Explain what it means for a function to be 0(1). 

31. Explain w hat it means lor a function to be 0(1) 


32. Give a big-0 estimate of the product of the first n odd 
positive integers. 

33. Show that if / and g are real-valued functions such 
that /<: x i is 0(g(x)), then /*fx) is Oigfx)). [Mote that 
/ fc W = fUf, J 

34. Show that if f{x) is Ollog, ,r) where b > I, then fix) 
is Oilog,, x) where a > I. 

35. Suppose that fix) is Oig(x)) where/ and g are increas¬ 
ing and unbounded functions. Show that log j/(x)i is 
G(!og g(t)l), 

36. Suppose that fix) is Oig(x)). Does it follow that 2 f,J> 
kOi :' 1 

37. Let f(.x) and /j(x) be functions from the set of real 
numbers to the set of positive real numbers. Show that 
if fix) and fix) are both BigU)), where gix) is a 
function from the set of real numbers to the set of posi¬ 
tive real numbers, then fix) 4 fix) is 0(g(v)j, Is this 
still true if fix) and fiix) can take negative values? 

38. Suppose that/fxj, g(x), and h(x) are functions such that 
fix) is 0(g(J£)) and g(x) is 0ffiU)j. Show (hat fix) is 
0i hi v; i 

39. ]f / (jv) and fix) are functions from the set of positive 
integers to the set of positive real numbers and fix) 
and f (x) are both 0(g (x)l, is (f - f )(» also 0(g(x))? 
Either prove that it is or give a counterexample, 

40. Show that if f(x) and fix) are functions from the 
set of positive integers to the set of real numbers and 
Mx) is ®(g] (x)) and fix) is <-)■' cv}), then ifffi.x) 
is ©tgi £>(*)}, 

41. Find functions/ and g from the set of positive integers 
tt> tile set of real numbers such that /{«> is not Oiginn 
and gin) is not Oifin)) simultaneously, 

42. Express the relationship fix) is fl(g(j)) using a pic¬ 
ture. Show the graphs of the functions /fx) and Cgix ). 
as w r ell as the constant k on the real axis. 

43. Show that if /|(x) is ffgitx)), fix) is 0(g ? (.r)), and 
f(x) 4 s 0 and g 2 (x) ¥> 0 for ail real numbers x > 0, 
then ififfix) is 0((g 1 /g 2 )(,r}), 

44. Show that if f(x) = a„x* 4 n„_, V 1 4 - ■ ■ 4 o, ,y + 

where no, n,_ u „-\, a„ arc real numbers and a„ * 0. 

then f[x) is 0(x“). 

Big-O, big-Theta, and big-Otnega notation can be extended 
to 1 unctions in more than one variable. For example, the 
statement fix. y) is 0{g(x. y)i means that there exist con¬ 
stants C, k\. and f such that [/(x,y)l 5 Cjg(x. v)| when¬ 
ever v > k [ and v > k 2 . 

45. Define the statement fix, v) is 0(g(v, v(). 

46. Define the statement fix, y) is ft(g(jt, y) J. 

47. Show that fa 2 4 xy 4 x Wgy) 3 is i.tir v’ t. 

48. Show that v- v 2 4 v 4 v 4 + , r y is ftf rV'). 

49. Show that [xyj is Of.ty). 

56, Show thatf.ry? is fffxv), 

The following problems deal with another type of asymp¬ 
totic notation, called little-o notation. Because httie-o 
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notation is based on the concept of limits, a knowledge of 
calculus is needed lor these problems. We say that /(x) is 
cn.vd [mad /(.»■) is 'little-oh” of g(x)], when 


51. [Calculus required) Show' that 

a) v‘ •> ?)( •.•'i h) t log x is a{ x - ) 

c) .i' is o{2'} d) .r - x + 1 is not o(jr) 

52. t Calculus required) 

a) Show that if /(x) and g(xj are functions such that 
ft 0 is w(g(jr)l and c is a constant, then cf(. v) is 
«(gtx)) where u-f){x) = cf{x). 
h) Show that if / L {jc>. fy 11 , and g(x) are functions 
such that / L (x) is and /;(.r) is «(g(x)), then 

(f\ +/:)u)is »(g(xi), where ij) +fz)(x) = /i(x) + 
/tl-r). 

53, (Calculus required) Represent pictorially that x log x is 
»(.sr) by graphing xlogx, .r, and xlogx/x 2 . Explain 
how this picture show's that x log a is p(jr). 

54, (Calculus required) Express the relationship fix) is 
o a >t using a picture. Show the graphs of /(.r), g{.r). 
and fU)!g[x). 

*55. (Calculus required) Suppose that fix) is n:g:.t)) Does 
it follow that 2 m is n(2«)? 

*56. (Calculus required) Suppose that fix) is oig(x)). Docs 
it follow' thut log |/(.r)| is o(!og[gU)|)‘.' 

57. (Calculus required) The two parts of this exercise de¬ 
scribe the relationship between little-a and big-0 nota¬ 
tion. 

a) Show that if /lx) and g{x) are functions such that 
fit i is ri(i’(x)), then fix) is 0(g(x)). 

b) Show that if fix) and g(.r) are functions such that 
fix t is 0(g(x),L then it does not necessarily follow 
that /(r) is w(g(.v|). 


58. (Calculus required) Show that if fix) is a polynomial 
of degree n and g(x] is a polynomial of degree m w here 
m > n , then /(x)is o(g(x)). 

59. (Calculus required) Show that if f(x) is ()(g(x)j turd 
f 2 (x) is ulj(x)), then fix) + fix) is 0 (jfx)j. 

60. (Calculus required) Let H n be the nth harmonic num¬ 
ber 


Show that H h is Ofiog n). (Hint: First establish the in¬ 
equality 



by showing that die sum of the areas of the rectangles of 

height Uj with base from j - 1 to j. for j = 2 . 3 ,_ n, 

is less than the area under the curve v - l/.r from 
2 to n.) 

*61, Show that n log n is 0(log n\). 

62. Determine whether logi n!) is Oin log n). Justify your 
answer. 

Let /lx) and j?Ct)be functions from die set of real numbers 
to the set of real numbers. We say that / and g are asymp¬ 
totic and write fix) — jr(v) if lim,-,. f(x)igix) -- 1. 

63. (Calculus required) For each of die following pairs of 
functions, determine whether/ and g are asymptotic, 
a) f(x) = x 2 + 3x + 7 ,g(x) = .v 2 + 10 

hi fix) - v log g( >.i = .C 

e) fi.X) = x J + tog(3x* + 7). g(x) - (x J 1 I7.v + 3t : 
d) f{x) = ix' + x - x + 1 ) 4 , g(jr) - (x 4 F x- + x 7 + 

X t I) 3 . 

el f(x) - log(x 2 -(- 1), jf(x) logx 

f) fix) = 2* +3 ,g(jt) - 2* +1 

g) fix) 2 : \g(.x) - 2 ' 


Key Terms and Results 


LOGIC (SECTIONS 1-3): 

TERMS 

proposition: a statement that is true or false 
truth value: true or false 

ip t negation of pi: the proposition with truth value op¬ 
posite to the truth value of p 

logical operators: operators used to combine proposi¬ 
tions 


compound proposition: a proposition constructed by 
combining propositions using logical operators 
truth table: a table displaying the truth values of propo¬ 
sitions 

p V 9 (disjunction of p and 9 ): the proposition that i s true 
unless both p and q are false 

p A q (conjunction of p and 9 ): the proposition that is true 
only when both p and q are true 
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p f.«/ (exclusive or of p and q); the proposition that is true 
when exactly one ofp and q is true 
p q (p implies q)\ the proposition that is false only 
when p is true and q is false 
converse of p q: the implication ij —*■ p 
contrapositive of p —^ the implication q ^ -<p 

p *■-* q ( biconditional); the proposition that is true only 
when p and q have the same truth value 
bit: either a 0 or a 1 

Boolean variable: a variable that has a value of 0 or 1 
bit operation: an operation on a bit or hits 
bit string: a list of bits 

bitwise operations: operations on bit strings that operate 
on each bit in one string and the corresponding bit in the 
other string 

tautology: a compound proposition that is always true 
contradiction: a compound proposition thaL is always 
false 

contingency: a compound proposition that is sometimes 
true and sometimes false 


SETS (SECTIONS 4-5): 


TERMS 

set: a collection of distinct objects 
axiom: a basic assumption of a theory 
paradox: a logical inconsistency 
element, member of a set: an object in a set 
0 (empty set, null set): the set with no members 
universal set: the set containing all objects under consid¬ 
eration 

Vent) diagram: a graphical represeniation of a set or sets 
S = T (set equality): S and T have the same elements 
S C T iS is a subset of 71: every element of 5 is also an 
element of T 

-S’ C T (S is a proper subset of 71: .Visa subset of T and 

S + T 

Unite set: a set with n elements where ji is a nnnnegative 
integer 

infinite set: a sei (hat is not finite 


( UNCTIONS (SECTIONS 6-8): 


TERMS 

function from A to B: an assignment of exactly one ele¬ 
ment of B to each element of A 
domain of J : the set 4 where/ is a function from A to B 


logical equivalence: compound propositions are logic¬ 
ally equivalent if they always have the same truth 
values 

propositional function: the combination of a variable and 
a predicate 

universe of discourse: the domain of the variable in a 
propositional function 

3.v P\x) (existential quantification of P(x}): the propo¬ 
sition that is true if and only if there exists an x in the 
universe of discourse such that P(x) is true 

Vi Pi x) (universal quantification of P(x)); the proposi¬ 
tion that is true if and only if P(x) is true for all x m the 
universe of discourse 

free variable: a variable not bound in a propositional 
function 

RESULTS 

The logical equivalences given in Tables 3 and 6 in Sec¬ 
tion 2 


iSj (the cardinality of S): the number of elements m 5 
P(5) (the power set of S): the set of all subsets of S 
A U H (the union of /I and B): the set containing those 
elements that are in at least one of A and B 
.4 H B (the intersection of A and B)r the set containing 
those dements that are in both 4 and B. 

4-6 (the difference of .4 and D): the set containing those 
elements that are in 4 but not in B 
4 (the complement of 4); the set of elements in the uni¬ 
versal set that are not in 4 

4 0 B (the symmetric difference of A and By, the seL 

containing those dements in exactly one of A and B 
membership table: a table displaying the membership of 
elements in sets 

RESULTS 

The set identities given in Table 1 in Section 5 


codomain of/: [he set B where / is a function from 4 

to 8 

b is the image of a under /: b ■ f(a) 
a is a pre-image of h u nder /: f (a ) ~ b 
range of/: the set of images of/ 
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onto function, surjection. 1 a function from .4 to B such that 
ever;' element of B is the image of some element in A 
one-to-one function, injection: a function such that the 
images of elements in its domain are all different 
one-to-one correspondence. Injection: a function that is 
both one-to-one and onto 

inverse of f : the function that reverses the correspondence 
given by/ (when/ is a bijection) 
fog (composition of / and »): the function that assigns 
fig(x)) to -v 

[j?j (floor function): the largest integer not exceeding r 
Lrl (ceiling function): the smallest integer greater than or 
equal to x 

sequence: a function with domain that is a subset of the 
set of integers 
string: a finite sequence 
ff“- j a,: the sum at * iti + ■■ ■ 4 a„ 

J If— f <3,: the product rtrdtjfi • ■ a„ 


countable sett a set that is either finite or that can be 
placed in one-to-one correspondence with the set of pos 
itive integers 

uncountable set: a set that is not countable 
fix) is 0(g(x))i the fact that |/(x)| C\g(x)\ for all x > k 

for some constants C and k 

/(.t)isfl(g(»): the fact that |/(.tj| a C\g(x)\ for all x > k 
for some positive constants C and k 
fix) is 0(g(x)): the fact that f{i) is 0(g(x)) and fix) is 

RES| ITS 

The set of real numbers is uncountable, 
log n 1 is 0(>i lug n). 

if/i(Jt) is 0(g,(x)) and fix) is 0(g 2 (x)), then (/, + / 2 )(c) 
is Otmaxte u;. and (/, fhxi is 0(g,(x)g 2 (x)). 

If Od.ai. a„ are real numbers, then of 4 

■ ■ - + ni .t + oq is C){x n ) and Of*"). 


Review Questions 


1. a) Define the negation of a proposition. 

bl What is the negation of “This is a boring course”'.’ 

2. a) Define (using truth tables) the disjunction, conjunc¬ 

tion, exclusive or, conditional, and biconditional of 
the propositions p and q. 

b) What arc the disjunction, conjunction, exclusive or, 
implication, and biconditional of the propositions 
‘Til go to the movies tonight” and “I'll finish my 
discrete mathematics homework”',’ 

3. a) Describe at least five different ways to wri te the im¬ 

plication p —* q in English, 
b) Define the converse and eontrapasitive of an impli¬ 
cation. 

cl State the converse and the contrapositive of the im¬ 
plication “If it is sunny tomorrow, then I will go for 
a walk in the woods,” 

4. a) What docs ft mean for two propositions to be logi¬ 

cally equivalent’ 

b) Describe the different ways to show that two com¬ 
pound propositions are logically equivalent, 
r) Show in at least two different ways that the com¬ 
pound propositions ~^p\/(r —* -i q) and -ip\/ 
-i q v 11 ' ure equivalent. 

5 . {Depends mi the' Exercise Set in Section 1,2} 

a) Given a truth table, explain how to use disjunctive 
norma! form te construct a compound proposition 
with this truth table. 

h) Explain w hy part (a) shows that the operators A, y, 
and -i are functionally complete. 


c) Is there an operator such that the set containing just 
this operator is functionally complete? 

6. What are the universal and existential quantifications 
of a predicate Pi. xff What are their negations? 

7. a) What is the difference between the quantification 

3.r V> P(x, vi and Vy T v p(x, y), w here Fix, y) is a 
predicate? 

b) Give an example of a predicate pi v, y) such that 
3 r Vy /’( v, y) and Vv3.r P\X, y) have different 
truth values. 

8 . a) Define the union, intersection, difference, and sym¬ 

metric difference of two sets. 

b) What are the union, intersection, difference, and 
symmetric difference of the set of positive integers 
and the set of odd integers? 

9. a) Define what it means for two sets to be equal. 

bl Describe the ways to show that two sets are 
equal. 

c) Show in at least two different ways that the sets 
A - (B n C) and (A - fit U (A - Cl are equal. 

10. Explain the relationship between logical equivalences 
and set identities. 

11. a) Define |3"|, the cardinality of the set S. 

h} Give a formula for \A U B\ where A and B are 
sets, 

12. a) Define the power set of a sei S. 

b) When is the empty set in the power set of a set P 
e) Flow many elements does the power set of a set 5 
with n elements have? 
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13, a) Define the domain, codomain, and the range of a 

function. 

b) Let f(n) be the function from the set of integers to 
the set of integers such that fin) = n 2 1 - 1. What 
are the domain, codomain, and range of this func¬ 
tion? 

14, a) Define what it means for a function from the set of 

positive integers to the set of positive integers to be 
one-to-one. 

b) Define what it means for a function from the set of 
positive integers to the set of positive integers to be 

onto. 

c) Give an example of a function from the set of pos¬ 
itive integers to the set of positive integers that is 
both one-to-one and onto. 

d) Give an example of a function from the set of pos¬ 
itive integers to the set of positive integers that is 
one-to- one but not onto. 

e) Give an example of a function from the set of pos¬ 
itive integers to the set of positive integers that is 
not one-to-one but is onto. 

f) Give an example of a function from the set of pos¬ 
itive integers to the set of positive integers that is 
neither one-to-one nor onto. 

15. at Define the inverse of a function. 

b) When does a function have an inverse? 
el Does the function f(n) = 10 - n from the set of 
integers to the set of integers have an inverse? If 
so, what is it? 

16. a) Define the floor and ceiling functions from the set 

of real numbers to the set of integers. 


b) For which real numbers ,r is it true that [.rj = [xl? 

17, a) Use summation notation to express the sum of the 

powers of 2 from 2° to 2". 
b) What is the value of the sum in part (a)? 

18, a) What does it mean for a set to be countable? Give a 

precise definition. 

b) fs the set of negative integers countable? Why or 
why not? 

c) Is the set of rational numbers with denominators 
greater than 3 countable? Why or why not? 

d) Is the set of real numbers between 2 and 3 count¬ 
able? Why or why not'? 

19, a) State the definition of the fact that f(n) is 0(g(n)), 

where f(n) and gin) are functions from the set of 
positive integers to the set of real numbers, 

b) Use the definition of the fact that fin) is 0{g(n}\ 
directly to prove or disprove that n 2 + I Sn + 107 is 
0(n :i ). 

c) Use the definition of the fact that fin) is 0(g{n}) 
directly to prove or disprove that n 3 is Oirf + 18rt + 
107). 

20, a) How can you produce a big-0 estimate for a func¬ 

tion that is the sum of different terms where each 
term is the product of several functions ? 
b) Give a bsg-0 estimate for the function fin) = 
(a! + 1)(2" + 1) + {n"' ~ + U"- 2 )(n } + 2"). 
For the function g in your estimate fix) is 
D(g(.t)1 use a simple function of smallest possible 
order. 


Supplementary Exercises 


1. I .et p be the proposition "I will do eveiy exercise in this 
book" and q be the proposition “I will get an ‘A’ in this 
course." Express each of the following as a combina¬ 
tion of p and q. 

a) I will gel an ‘A’ in this course only if I do every 
exercise in this book. 

b) l will get an 'A' in this course and I will do every 
exercise in this book. 

c) Either I will not get an ‘A‘ in this course or I will 
not do every exercise in this book. 

d) For me to get an 'A’ in this course it is neces¬ 
sary and sufficient that ! do every exercise in this 
book. 

2. Find the truth table of the compound proposition 

(pVq) — (p A —. r). 


3. Show that the following propositions are tautologies. 

a) (19 A (p -> q) \ -*■ -1 p 

b) UpVq) A -p) —> q 

4. Give the converse and the contrapositive of the follow¬ 
ing implications. 

a) If it rains today, then 1 will drive to work, 

b) If |jt| - x, then x a 0. 

c) If n is greater than 3, then n 2 is greater than 9. 

5. Find a compound proposition involving the proposi¬ 
tional variables p, q, r, and s that is true when exactly 
three of these propositional variables are true and is 
false otherwise. 

6. Let P{x\ be the statement “student ,r knows calculus" 
and let {?(>') be the statement “classy contains a student 
who knows calculus." Express each of the following as 
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quantifications of Ff.v) and (J- • s 

a) Some students know calculus 

b) Not every student knows calculus, 

ei Every class has a student in it who knows calculus. 

d) Every student in every class knows calculus. 

e) There is at least one class with no students who 
know calculus. 

7. Let P(m, n\ be the statement “m divides n." where the 
universe of discourse for both variables is the set of pos¬ 
itive, integers Determine the truth values of each of the 
following propositions. 

a) P(4,5} b) E(2.4) 

c} Vm Vri Pl m, n) d) 3m V ti P{m, n) 

e) 3/iVm P(m, n) f) V/» F'f I ni 

8. Let /'t v y) he a propositional function. Show that the 
implication 3.rVv/ , (r, v) -* Vy 3xP(x, y) is a tautol¬ 
ogy. 

9. Let PU, and Q{. r). be propositional functions. Show' 
that 3 a ( Fix) • Q\x)) and V.t POr) — 3.v Q(x) al- 
tray‘ have the same truth value. 

11). If Vy 3.v P{ x. v) is true, does it necessarily follow dial 
3.i Vv P( t. y) is true? 

11. If V.v 3 v Pix, v) is true, does it necessarily follow that 
3 t Vy P(x, rj is true'.’ 

12. Find the negations of the following statements. 

a t if it snows today, then I will go skiing tomorrow. 

b) F.very person in this class understands mathemati¬ 
cal induction. 

el twine students in this class do not like discrete 
mathematics. 

d) In every mathematics class there is some student 
who fails asleep during lectures. 

13. Express the following statement using quantifiers: 
"Every student in this class has taken some course 
in every department in the school of mathematical 
sciences." 

14. F\p ress the following statement using quantifiers: 
“There is a building nn the campus of some college 
in the United Slates in which every room is painted 
white." 

15. Let A be the set of English words that contain the letter 
a , and let B be the set of English words that contain the 
letter tf, Express each of the following sets as a combi¬ 
nation of 4 and 5, 

a) The set of English words that do not contain the let¬ 
ter i. 

b> The set of English words that contain both an ,r 
and a q. 

cl The set of English words that contain an .r but 
not a q. 

d) The set of English words that do not contain either 
an x or a q. 

e) The set of English words that contain an ,t or a q H 
but not both. 


16. Show that if A is a subset of 25, then the power set of A 
is a subset of the power set of B. 

17, Suppose that A and B are sets such that the power set of 
4 is a subset of the power set of B Does it follow that 
A ts a subset of B ! 

18, Let E denote the set of even integers and O denote the 
set of odd integers. As usual, let Z denote the set of all 
integers. Determine each of the following. 

a) E U O b) E n O c) Z - E d) Z - O 

19. Show that if A is a set and U is the universal set, then 
a) ,4 n A 0 . b) .4 u A = U. 

20, Show that if A and B are sets, then 

a) A A n (4 U Si. b) A = 4 U (4 H it). 

21, ShnwthatifA and 25 are sets, then A-(.4-/5) = AVtB. 

22. Let A and B be sets. Show that A C B if and only if 
A n B =- A 

23, Let A, B, and C be sets. Show that (A - Bt - C is rot 
necessarily equal to A - (B - Ct. 

24, Suppose that 4, B and C are sets. Prove or disprove 
that (4 -B)- C = (A - C) -if. 

25. Suppose that A, fi, C. and D are sets. Prove or disprove 
that (A - B) - (C - D) = (A - C) - (B - D). 

26. Show that if A and B are finite sets, then \A n B\ s 
i A UBj. Determine when this relationship is an equality. 

27, Let 4 and B he sets in a finite universal set U. 1.1st the 
following in order of increasing size. 

a) 141, \A U B\ [4 n B|, |{/|, 10 i 

b) 14 - B\, |,4 * B\, f4j -r | SI, |4 U B\,} 01 

28, Let A and B be subsets of the finite universal set V. 
Show that jA D B\ = (L’l - J4| - |B| -E (4 0 B\ 

29. Let / and g he functions from {1, 2. 3,4} to {a, b, r, d) 
and from {a,b,c,d} lo (1,2,3, 4), respectively, such 
that /CD - d. fQ) c. ft 3) = a, fiA) h. and 
gia) - 2, gib) = 1, g(c) = 3, g(tl) = 2. 

a) Is / one-to-one? Is g one-to-one? 

b) h f onto? Is g onto? 

c) Does either/or g have an inverse? If so, find this 
inverse. 

30. Let/ be a one-to-one function from the set 4 to the set 
B. Lei S and T be subsets of A. Show that fiS D T) ■ 

m n 

31. Give an example to show that the equality in Exercise 
30 may not hold iff is not one-to-one. 

32, Show that if n is an integer, then n ~ \n/2) t [w/ 2 j. 

33. Find the value of the following quantities. 


b) II 


Lj 


c) 



d,r i(,v) 


34, Is the set of irrational numbers between 0 and I count¬ 
able? Justify your answer 
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**35. A real number is called algebraic if it is the root of a 
polynomial with integer coefficients. Show that there 
are a countable number of algebraic numbers, {Hint: 
Use the fact that a polynomial of degree n has at most 
a distinct roots,} 


M. Show that 8 c : + I2 ji + 100 log x is Q(.r 3 ). 

37. Give a big-0estimate for(.r + x (log a) 3 ) ■ (2* + .ri 

38. Find a big-G estimate for ^" J = i jij + 1) 

*39. Show that h! is not 0(2"). 

*40. Show that n a is no: (An!). 


Computer Projects 

V-.il . I PROGRAMS WITH THE SPECiflER INPUT AND OUTPUT. 


1. Given the truth values of the propositions p and a, find 
the truth values of the conjunction, disjunction, exclu¬ 
sive on implication, and biconditional of these propo¬ 
sitions. 

2. Given two bit strings of length n. find the bitwise AND, 
bitwise OR, and bitwise XOR of these strings. 

3. Given the truth values of the propositions p and q in 
fuzzy logic, find the truth value of the disjunction and 
the conjunction of p and q (see Exercises 31-33 ofSec- 
tion 1.1). 

4. Given subsets A and B of a set with rt elements, 
use hit strings to find 4. A U B, A n B t A - fi, and 

•• ir H 


5. Given multisets A and B from the same universal set, 
find A u if, ,4 fl B, A - B, and A + B (see preamble to 
Exercise 47 of Section 1,5). 

6. Given fuzzy sets 4 and B, find 4, A U B. and A n 
(see preamble to Exercise 49 of Section 1.5). 

7. Given a function/ from [1,2. n} to the set of inte¬ 

gers, determine whether/ is one-to-one. 

8. Given a function/ from {1, 2__ n \ to itself, determine 

whether/ is onto, 

9. Given a bijection/ from the set {1,2,. ;i! to itself, 

find /-'. 

10, Given the terms of a sequence aj, u>,.. n„, find 
>.• c,.md H* =] fly. 


Computations and Explorations 

l 'SI- A tOMPUTATlONAI. PROGRAM OR PROGRAMS YOU HAVE WRITTEN TO DO THE FOLLOWING EXERCISES. 


1, Wh;a is the largest value of n for which n\ has fewer 
than 100 decimal digits and fewer than 1000 decimal 
digits? 

2, Ht»w many zeros are there at the end of the decimal rep¬ 
resentation of «! for each of the first 25 positive inte¬ 
gers n? Can you figure out a formula for the number 
of zeros at the end of ttl in decimal nutation? (See Sec¬ 
tion 2.3.) 

3, Calculate the number of one-to-one functions front a 
set S to a set T, where S and T are finite sets of var¬ 
ious sizes. Can you determine a formula for the num¬ 


ber of such functions? (We will find such a formula in 
Chapter 4.) 

4. Calculate the number of omo functions from a set S to a 
set T where S and T are finite sets of various sizes. Can 
you determine a formula for the number of such func¬ 
tions? (We will find such a formula in Chapter 5.) 

5. We know that n h is 0(d n ) when b and dare positive num¬ 
bers with J s 2, Give values of the constants C and k 
such that n b r£ Cd” whenever x> k for each of the fol¬ 
lowing sets of values: b = 10, d = 2; b = 20, d = 3; 
b = 1000, d = 7. 


Willing Projects 

RESPOND TO THE FOLLOWING WITH ESSAYS USING OUTSIDE SOURCES. 


1, Describe how fuzzy logic is being applied to practical 
applications. Consult one or more of the recent books on 
fuzzy logic written for general audiences. 


2. Read some of the writings of Lewis Carroll on symbolic 
logic. Describe in detail some of the models he used to 
represent logical arguments. 
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3. Discuss how an axiomatic set theory can be developed 
to avoid Russell's paradox. (See Exercise 26 of Section 
1.4.) 

4. Research where the concept of a function first arose, and 
describe how this concept was first used. 

5. Explain how various people have found The Encyclope¬ 
dia of Integer Sequences [SIPJ95] useful. Also, describe 
a few of the more unusual sequences in this encyclope¬ 
dia and how they arise. 


6- Describe how the concept of the cardinality of sets can 
be extended to infinite sets. 

7. Look up the definition of a transcendental number. Ex¬ 
plain how to show that such numbers exist and how such 
numbers can be constructed. Which famous numbers 
can be shown to be transcendental? 

8, Look up Bach man ns original introduction of big-t) no¬ 
tation. Explain how he and others have used this nota¬ 
tion. 
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The Fundamentals: 
Algorithms, the Integers, 
and Matrices 



M any problems can be solved by considering them as special cases of general 
problems. For instance, consider the problem of locating the largest integer 
in the sequence 101, 12, 144,212, 98. This is a specific case of the problem 
of locating the largest integer in a sequence of integers. To solve this general problem 
we must give an algorithm, which specifies a sequence of steps used to solve this gen¬ 
eral problem. We will study algorithms for solving many different types of problems in 
this book. For instance, algorithms will be developed for finding the greatest common 
divisor of two integers, for generating all the orderings of a finite set, for searching a 
list, and for finding the shortest path between two vertices in a network. One impor¬ 
tant consideration concerning an algorithm is its computational complexity. That is, 
what are the computer resources needed to use this algorithm to solve a problem of a 
specified size? We will illustrate the analysis of the complexity of algorithms in this 
chapter. 

The set of integers plays a fundamental role in discrete mathematics. In particu¬ 
lar, the concept of division of integers is fundamental to computer arithmetic. We will 
briefly review some of the important concepts of number theory, the study of integers 
and their properties. Some important algorithms involving integers will be studied, in¬ 
cluding the Euclidean algorithm for computing greatest common divisors, which was 
first described thousands of years ago. Integers can be represented using any positive 
integer greater than 1 as a base. Binary expansions, which are used throughout com¬ 
puter science, are representations with 2 as the base. In this chapter we discuss base 
b representations of integers and give an algorithm for finding them. Algorithms for 
integer arithmetic, which were the first procedures called algorithms, will also be dis¬ 
cussed. This chapter also introduces several important applications of number theory. 
For example, in this chapter we will use number theory to make messages secret, to 
generate pseudorandom numbers, and to assign memory locations to computer files. 
Number theory, once considered the purest of subjects, has become an essential tool in 
providing computer and Internet security. 

Matrices are used in discrete mathematics to represent a variety of discrete struc¬ 
tures. We review the basic material about matrices and matrix arithmetic needed to 
represent relations and graphs. Matrix arithmetic will be used in numerous algorithms 
involving these structures. 

2.1 ___ __ 

Algorithms 


INTRODUCTION 


There are many general classes of problems that arise in discrete mathematics. For 
instance: given a sequence of integers, find the largest one; given a set, list all of its 

99 
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subsets; given a set of integers, put them in increasing order; given a network, (inti the 
shortest path between two vertices. When presented with such a problem, the first thing 
to do is to construct a model that translates the problem into a mathematical contest. Dis¬ 
crete structures used in such models include sets, sequences, and functions—structures 
discussed in Chapter 1—as well as such other structures as permutations, relations, 
graphs, trees, networks, and finite state machines—concepts that will be discussed in 
later chapters. 

Setting up the appropriate mathematical model is only part of the solution. To com¬ 
plete the solution, a method is needed that will solve the general problem using the 
model. Ideally, what is required is a procedure that follows a sequence of' steps that 
leads to the desired answer. Such a sequence of steps is called an algorithm. 


DEFINITION 1, An algorithm is a fini te set of precise instructions for performing 
a computation or for solving a problem. 


The term algorithm is a corruption of the name al-Khowarmm. an Arabian math¬ 
ematician of the ninth century, whose book on Hindu numerals is the basis of modem 
decimal notation. Originally, the word algorism was used for the rules for performing 
arithmetic using decimal notation. Algorism evolved into the word algorithm by the 
eighteenth century. With the growing interest in computing machines, the concept of 
an algorithm was given a more general meaning, to include all definite procedures for 
solving problems, not just the procedures for performing arithmetic. {We will discuss 
algorithms for performing arithmetic with integers in Section 2.4.) 

In this book, we will discuss algorithms that solve a wide variety of problems. In 
this section we will use the problem of finding the largest integer in a finite sequence 
of integers to illustrate the concept of an algorithm and the properties algorithms have. 
Also, we will describe algorithms for locating a particular element in a finite set. in 
subsequent sections, procedures for finding the greatest common divisor of two integers, 
for finding the shortest path between two points in a network, for multiplying matrices, 
and so on, will be discussed. 


EXAMPLE 1 ’ Describe an algorithm for finding the maximum (largest) value in a finite sequence of 

integers. 

Even though the problem of finding the maximum element in a sequence is rel¬ 
atively trivial, it provides a good illustration of the concept of an algorithm. Also, 
there are many instances where the largest integer in a finite sequence of integers is 
required. For instance, a university may need to find the highest score on a competitive 
exam taken by thousands of students. Or a sports organization may want to identity the 


Ahu Ja'far Mnhammed ibn Musa al-Kbowarizmi (c, 850). al-Khowarizmi, an astronomer and 

mathematician, was a member of the House of Wisdom, an academy of scientists in Baghdad. The name 
W€b al-Kbowarizmi means "from the town of Kowarztzm," which is now called Khiva and is part of i .Vbekisiart, 
al-khowanzmi wrote books on mathematics, astronomy, and geography, Western Europeans first learned 
about algebra from his works. The word algebra comes from al-jabr. part of (he title of hi* book Kituh ul-mbr 
v ' at muqttaMa. This book was translated into Latin and was a widely used textbook. Lbs book on the use 
of Hindu numerals describes procedures for arithmetic operations using these numerals, European authors 
used a Latin corruption of his name, which later evolved to the word algorithm to describe the subject ot 
arithmetic with Hindu numerals. 
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member with the highest rating each month. We want to develop an algorithm that can 
be used whenever the problem of finding the largest element in a finite sequence of 
integers arises. 

We can specify a procedure for solving this problem in several ways. One method 
is simply to use the English language to describe the sequence of steps used. We now 
provide such a solution. 

Solution of Example I: We perform the following steps. 

1. Set the temporary maximum equal to the first integer in the sequence. ( The tempo¬ 
rary maximum will be the largest integer exa min ed at any stage of the procedure.) 

2. Compare die next integer in the sequence to die temporary maximum, and if it 
is larger than the temporary maximum, set the temporary maximum equal to this 
integer. 

3. Repeat the previous step if there are more integers in the sequence. 

4. Stop when there are no integers left in the sequence. The temporary maximum at 

this pint is the largest integer in the sequence. ■ 


An algorithm can also be described using a computer language. However, when 
that is done, only those instructions permitted in the language can be used. This often 
leads to a description of the algorithm that is complicated and difficult to understand. 
Furthermore, since many programming languages are in common use, it would be un¬ 
desirable to choose one particular language. So, instead of using a particular computer 
language to specify algorithms, a form of pseudocode will be used in this book. (All 
algorithms will also be described using the English language.) Pseudocode provides an 
intermediate step between an English language description of an algorithm and an im¬ 
plementation of this algorithm in a programming language. The steps of the algorithm 
are specified using instructions resembling those used in programming languages. How¬ 
ever, in pseudocode, the instructions used can include any well-defined operations or 
statements. A computer program can be produced m any computer language using the 
pseudocode description as a starting point. 

The pseudocode used in this book is loosely based on the programming language 
Pascal. However, the syntax of Pascal, or that of other programming languages, will not 
be followed. Furthermore, any well-defined instruction can be used in this pseudocode. 
The details of the pseudocode used in the text are given in Appendix 2. The reader 
should refer to this appendix whenever the need arises. 

A pseudocode description of the algorithm for finding the maximum element in a 
finite sequence follows. 


ALGORITHM i Finding the Maximum Element in a Finite Sequence. 

procedure m<ZK{a h co, ..., a n \ integers) 
max: = Hi 
for i : = 2 to n 

if max < a, then max : = n, 

{max is the largest element} 
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This algorithm first assigns the initial term of the sequence, a\ , to the variable max. The 
“for” loop is used to successively examine terms of the sequence. If a term is greater 
than the current value of max, it is assigned to be the new value of mat. 

There are several properties that algorithms generally share. They are useful to 
keep in mind when algorithms are described. These properties are: 

■ Input, An algorithm has input values from a specified set, 

■ Output. From each set of input values an algorithm produces output values from a 
specified set. The output values are the solution to the problem. 

■ Definiteness. The steps of an algorithm must he defined precisely. 

■ Correctness. An algorithm should produce the correct output values for each set 
of input values. 

■ Finite ness. An algorithm should produce the desired output after a finite (but per¬ 
haps large) number of steps for any input in the set. 

■ Effectiveness, It must be possible to perform each step of an algorithm exactly and 
in a finite amount of time. 

■ Generality. The procedure should he applicable for all problems of the desired 
form, not just for a particular set of input values. 


EXAMPLE 2 Show' that Algorithm 1 for finding the maximum element in a finite sequence of integers 

has all the properties listed. 

Solution: The input to Algorithm 1 is a sequence of integers. The output is the largest 
integer in the sequence. Each step of the algorithm is precisely defined, since only 
assignments, a finite loop, and conditional statements occur. The algorithm uses a finite 
number of steps, since it terminates after all the integers in the sequence have been 
examined. The algorithm can be carried out in a finite amount of time since each step 
is either a comparison or an assignment. Finally, Algorithm f is general, since it can be 
used to find the maximum of any finite sequence of integers. ■ 


SEARCHING ALGORITHMS 


The problem of locating an dement in an ordered list occurs in many contexts. For in- 
Ww stance, a program that checks the spelling of words searches for them in a dictionary, 
which is just an ordered list of words. Problems of this kind are called searching prob¬ 
lems, We will discuss several algorithms for searching in this section. We will study 
the number of steps used by each of these algorithms in Section 2,2. 

The general searching problem can be described as follows: Locate an element x 

in a list of distinct elements a u tf;. a„. or determine that it is not in the list. The 

solution to this search problem is the location of the term in the list that equals x (that 
is. / is the solution if x = a,) and is 0 if x is not in the list. 

The first algorithm that we will present is called the linear search, or sequential 
search, algorithm. The linear search algorithm begins by comparing x and a,. When 
■r = au the solution is the location of a,, namely, I. When x A a,, compare x with a 2 
Ibjr = a;, the solution is the location of aj, namely, 2. When x A a;, compare x with 
Continue this process, comparing x successively with each term of the list until a 
match is found, where the solution is the location of that term, unless no match occurs. 
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EXAMPLE 3 


If the entire list has been searched without locating x, the solution is 0. The pseudocode 
for the linear search algorithm is displayed as Algorithm 2. 


algorithm 2 The Linear Search Algorithm. 

procedure linear searchix: integer, a\,ai,..., a n : distinct integers) 
/ := 1 

while (/ £ n and x ¥= a,) 
it— i+1 

if r s n then location : - i 
else location : = 0 

{location is the subscript of term that equals x, or is 0 if x is not 
found} 


We will now consider another searching algorithm. This algorithm can be used 
when the list has terms occurring in order of increasing size (for instance; if the terms 
are numbers, they are listed from smallest to largest ; if they are words, they are listed in 
lexicographic, or alphabetic, order). This second algorithm is called the binary search 
algorithm. It proceeds by comparing the element to be located to the middle term of 
the list. The list is then split into two smaller sublists of the same size, or where one 
of these smaller lists has one fewer term than the other. The search continues by re¬ 
stricting the search to the appropriate sublist based on the comparison of the element 
to be located and the middle term. In the next section, it will be shown that the binary 
search algorithm is much more efficient than the linear search algorithm. The following 
example demonstrates how a binary search works. 


To search for 19 in the list 

1 2 3 567 8 10 12 13 15 16 IK 19 20 22. 

first split this list, which has 16 terms, into two smaller lists with eight terms each, 
namely. 

12 3 5 6 7 8 10 12 13 15 1618 19 20 22. 

Then, compare 19 and the largest tents in the first list. Since 10 < 19, the search for 19 
can be restricted to the list containing the 9th through the 16th terms of the original list. 
Next, split this list, which has eight terms, into the two smaller lists of four terms each, 
namely, 

12 13 15 16 18 19 20 22. 

Since !6 < 19 /comparing !9 with the largest term of the first list) the search is re¬ 
stricted to the second of these lists, which contains the 13th through the 16th terms of 
the original list. The list 18 19 20 22 is split into two lists, namely, 

18 19 20 22. 

Since 19 is not greater than the largest term of the first of these two lists, which is also 
19, the search is restricted to the first list: 18 19, which contains the 13th and 14th terms 
of the original list. Next, this list of two terms is split into two lists of one term each: 
18 and 19. Since 18 < 19, the search is restricted to the second list: the list containing 
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the 14th term of the list, which is 19. Now that the search has been narrowed down to 
one term, a comparison is made, and 19 is located as the 14th term in the original list. 


We now specify the steps of the binary search algorithm. To search for the integer 
x in the list a\, a„, where a 2 < 05 < ■ ■ • < a n , begin by comparing x with the 

middle term of the sequence, a mi where m = [(n ])/2j. (Recall that [jtJ is the greatest 

integer not exceeding x.j If .x > a m , the search for x can be restricted to the second half 

of the sequence, which is a m+1 . a„. If .vis not greater than a m . the search for 

.v can he restricted to the first half of the sequence, which is a 2 . a m . 

The search has now been restricted to a list with no more than [«/2l elements. Using 
the same procedure, compare .x to the middle term of the restricted list. Then restrict the 
search to the first or second half of the list. Repeat this process until a list with one rerm 
js obtained. Then determine whether this term is .x. Pseudocode for the binary search 
algorithm is displayed as Algorithm X 


! ai.kokjthm i The Binary Search Algorithm, 

I procedure binary searchix : integer. a\, o 2 .... a n : increasing integers) 

f: — 1 ti is left endpoint of search interval! 
j: = tt {j is right endpoint of search interval} 

| while i < j j 

; begin 

; ">■■= L(f + y)/2j 

if x > a m then i : — m + 1 ! 

else j: - m \ 

end ' 

if s - a i then location .= i 

| else locution : = 0 ! 

i i 

i {location is the subscript of term equal to .v, or 0 if x is not found} j 

Algorithm 3 proceeds by successively narrowing down the pan of the sequence 
being searched. At any given stage only' the terms beginning with ij f mid ending with 
itj are under consideration. In other words, i and j arc the smallest and largest subscripts 
of the remaining terms, respectively. Algorithm 3 continues narrowing the part of the 
sequence being searched until only one term of the sequence remains. When this is 
done, a comparison is made to see whether this term equals x. 


Exercises 


1. Lbt ail the steps used by Algorithm i to find the max- 
inuiru of the list L 8, 12, 9, 11. 2, 14, 5, 10, 4, 

2. Determine which characteristics of an algorithm the 
following procedures have and which they lack, 

a) procedure dmthiein: positive integer) 
while ii > 0 
n : - 2 h 


b) procedure dividerr. positive integer) 
while niQ 
begin 
m ; = Mn 
n ; - n - 1 
end 
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e) procedure Mm[n\ positive integer) 
sum ; ■■■ (' f 
while ; 10 

sum : --- sum -t-1 

ti > procedure chooseiujx integers) 
i . - either a or f) 

3. Device ; in algorithm that finds the sum of all the inte¬ 
gers in ll list. 

4. Devise an algorithm to compute x". where x is a real 
number and « is an integer, (Hint: First give a proce¬ 
dure fur computing .v" when n is nonnegative by suc¬ 
cessive multiplication by x, starling with !, Then ex¬ 
tend this procedure, and use the fact that x~ n - l/V 
to compute .v" when n is negative.) 

5. Describe an algorithm that interchanges the values uf 
the variables ,v and y, using only assignments. What is 
the iiiininiuni number of assignment statements needed 
io do lilts.’ 

b. Dcscrib: an algorithm that uses only assignment state¬ 
ments that replaces the triple (c, r.;j with tv,;.r), 
What is the minimum number of assignment state¬ 
ments needed? 

7. l ist all [be steos used to search for 9 in the sequence- !. 
!, 4, s ft, jt. <). !; using 
a) a linear search. bt a bin ary search. 

S. List all tile: steps used to search for 7 in the sequence 
given in Exercise 7. 

9, Describe an algorithm that inserts an integer a m the 
appropriate position into the list up ai, of inte¬ 
ger? that are in increasing order 

Ift. Describe an algorithm for finding the smallest integer 
in a finite sequence of natural numbers. 

11. Describe an algorithm that locates the first occurrence 
of the largest element in a brute list of integers, where 
the integers in the list are not necessarily distinct. 

12. Describe an algorithm that locates the last occurrence 
ol the smallest element in a niitEe list of integers, where 
the integers in the list arc not necessarily distinct. 

I?. Describe an algorithm that produces the maximum, me¬ 
dian. mean, and minimum of a set of three integers. (The 
median ol a set of integers is the middle element in the 
list when these integers are listed in order of increasing 
sire The mean of a set of integers is the sum of the 
integers divided by the number of integers in the set.) 


14. Describe an algorithm for finding both the largest and 
the smallest integers in a finite sequence of integers. 

15. Describe an algorithm that puts the first three terms of 
a sequence of integers of arbitrary' length in increasing 
order- 

lb, Describe an algorithm to find the longest word m an 
English sentence (where a word is a string of letters 
and a sentence is a iist of words, separated by blanks). 

i 7. Describe an algorithm that determines w hether a func¬ 
tion from a finite set to another finite set is onto. 

18. Describe an algori thin that determines whether a func¬ 
tion from a Untie set to another finite set is one-to-one. 

19. Describe an algorithm that will count the number of 1 s 
in a bit string by examining each bit of the string to 
determine whether it is a 1 bit. 

20. Change Algorithm 3 so that the binary search proce¬ 
dure compares ,v to u„ at each stage of die algorithm, 
with the algorithm terminating if x = a m . What ad van 
tage does this version of the algorithm have? 

21. The ternary search algorithm locates an dement in a 
listol increasing integers by successively splitting the 
list into three sublets of equal (or as close to equal as 
possible) si/e, and restricting the search to the appro¬ 
priate piece. Specify the steps of this algorithm, 

22. Sped I y the steps of an algorithm that locales an ele¬ 
ment in a list of increasing integers by successively 
splitting the list into four sublists of equal (or as close 
la equal as possible) size, and restricting the search to 
the appropriate piece, 

23. A mode of a list of integers is an clement that occurs at 
least as often as each of the other dements Devise an 
algorithm that finds a mode in a list of nondecreasino 
integers. 

24. Devise an algorithm that finds all modes (defined in 
Exercise 23) in a list of nondccreasing integers. 

25. Devise an algorithm that finds the first term of a se¬ 
quence of integers that equals .some previous term in 
the sequence. 

26. Devise an algorithm that finds all terms of a finite se¬ 
quence of integers that are greater than the sum of all 
previous terms of (he sequence. 

27. Devise an algorithm that finds the first term of a se¬ 
quence of positive integers that is less than the imme¬ 
diately preceding term of the sequence 


2 . 2 _ 

Complexity of Algorithms 

INTRODUCTION 

When does an algorithm provide a satisfactory solution to a problem? First, it must 
always produce the correct answer. How this can he demonstrated will be discussed in 
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EXAMPLE I 


Chapter 3. Second, it should be efficient. The efficiency of algorithms will be discussed 
in this section. 

How can the efficiency of an algorithm be analyzed? One measure of efficiency is 
the time used by a computer to solve a problem using the algorithm, when input values 
are of a specified size. A second measure is the amount of computer memory required 
to implement the algorithm w hen input val ues are of a specified size. 

Questions such as these involve the computational complexity of the algorithm. 
An analysis of the time required to solve a problem of a particular size involves the time 
complexity of the algorithm. An analysis of the computer memory required involves the 
space complexity of the algorithm. Considerations of the time and space complexity' of 
an algorithm are essential when algorithms are implemented. It is obviously important 
to know whether an algorithm will produce an answer in a microsecond, a minute, or a 
billion years. Likewise, the required memory must be available to solve a problem, so 
that space complexity must be taken into account. 

Considerations of space complexity are tied in with the particular data structures 
used to implement the algorithm. Because data structures are not dealt with in detail 
in this book, space complexity will not be considered. We will restrict our attention to 
time complexity. 

The time complexity of an algorithm can be expressed in terms of the number 
of operations used by the algorithm when the input has a particular size. The opera¬ 
tions used to measure time complexity can be the comparison of integers, the addition 
of integers, the multiplication of integers, the division of integers, or any other basic 
operation. 

Tune complexity is described in terms of the number of operations required instead 
of actual computer time because of the difference in time needed for different computers 
to perform basic operations. Moreover, it is quite complicated to break all operations 
down to the basic bit operations that a computer uses. Furthermore, the fastest com¬ 
puters in existence can perform basic bit operations (for instance, adding, multiplying, 
comparing, or exchanging two bits) in !0“^ second (1 nanosecond), but personal com¬ 
puters may require 10“ fr second (1 microsecond), which is 1000 times as long, to do 
the same operations. 

We illustrate how to analyze the time complexity of an algorithm by considering 
Algorithm I of Section 2.1, which finds the maximum of a finite set of integers. 


Describe the time complexity of Algorithm I of Section 2.1 for finding the maximum 
element in a set, 

Solution: The number of comparisons will be used as the measure of the time com¬ 
plexity of the algorithm, since comparisons are the basic operations used. 

To find the maximum element of a set with n elements, listed in an arbitrary order, 
the temporary maximum is first set equal to the initial term in the list. Then, after a 
comparison has been done to de termine that the end of the list has not yet been reached, 
the temporary maximum and second term are compared, updating the temporary max¬ 
imum to tiie value of the second term if it is larger. This procedure is continued, using 
two additional comparisons for each term of the list—one to determine that the end 
of the list lias not been reached and another to determine whether to update the tem¬ 
porary maximum. Since two comparisons are used for each of the second through the 
nth elements and one more comparison is used to exit the loop when i = n + 1, exactly 
2(/i - 1) + 1 - 2/i - I comparisons are used whenever this algorithm is applied. Hence, 
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EXAMPLE 2 


EXAMPLE 3 


the algorithm for finding the maximum of a set of n elements has time complexity 0{n), 
measured in terms of the number of comparisons used. ■ 


Next, the time complexity of searching algorithms will be analyzed. 


Describe the time complexity of the linear search algorithm. 

Solution: The number of comparisons used by the algorithm will be taken as the mea¬ 
sure of the ttme complexity. At each step of the loop in the algorithm, two comparisons 
are performed—one to see whether the end of the list has been reached and one to com¬ 
pare the element.x with a term of the list. Finally, one more comparison is made outside 
the loop. Consequently, if x - a,-. 2i + 1 comparisons are used. The most comparisons. 
In + 2. are required when the clement is not in the list. In this case, 2 n comparisons 
are used to determine that x is not cij, for i — 1 . 2. .... n, an additional comparison is 
used to exit the loop, and one comparison is made outside the loop. So w hen, a is not in 
the list, a total of 2n + 2 comparisons are used. Hence, a linear search requires at most 
O(n) comparisons. ■ 


The type of complexity analysis done in Example 2 is a worst-case analysis. By 
the worst-case performance of an algorithm, we mean the largest number of operations 
needed to solve the given problem using this algorithm on input of specified size. Worst- 
case analysis tells us how many operations an algorithm requires to guarantee that it will 
produce a solution. 


Describe the time complexity of the binary search algorithm. 

Solution : For simplicity, assume there are n = 2 k elements in the list ai,a 2 _ a„. 

where k is a nonnegative integer. Note that k = log n, flf n. the number of elements 
in the list, is not a power of 2, the list can be considered part of a larger list with 
2 *+] elements, where 2* < n < 2 K>] . Here 2* +l is the smallest power of 2 larger 
than n.) 

At each stage of the algorithm, i and j, the locations of the first term and the last 
term of the restricted list at that stage, are compared to sec whether the restricted list 
has more than one term. If i < j, a comparison is done to determine whether.r isgreater 
than the middle term of the restricted list. 

At. the first stage the search is restricted to a list w ith 2 k " 1 terms. So far, two com¬ 
parisons have been used. This procedure is continued, using two comparisons at each 
stage to restrict the search to a list with half as many terms. In other 'words, two com¬ 
parisons are used at the first stage of the algorithm when the list has 2* elements, two 
more w hen the search has been reduced to a list with 2 i:_ 1 elements, two more when the 
search has been reduced to a list with 2*"" elements, and so on, until two comparisons 
are used when the search has teen reduced to a list with 2 1 ^ 2 elements. Finally, when 
one term is left in the list, one comparison, tells us that there are no additional terms left, 
and one more comparison is used to determine if this term is .v. 

Hence, at most 2k + 2 = 2 log n- 1- 2 comparisons are required to perform a binary 
search when the list being searched has 2* elements. (If a is not a power of 2, the original 
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EXAMPLE 4 


list is expanded to a list wish 2* +1 terms, where k - [log/t), and the search requires at 
most 2 [log n] 4- 2 comparisons.) Consequently, a binary search requires at most O(log n) 
comparisons. From this analysis it follows that the binary search algorithm is more 
efficient, in the worst case, than a linear search. ■ 


Another important type of complexity analysis, besides worst-ease analysis, is 
called average-case analysis. The average number of operations used to solve the 
problem over all inputs of a given size is found in this type of analysis. Average-case 
time complexity analysis is usually much more complicated than worst-case analysis. 
However, the average-case analysis for the linear search algorithm can be done without 
difficulty, as shown in Example 4. 


Describe the average-case performance of the linear search algorithm, assuming that 
the element x is in the list. 


Solution: There are n types of possible inputs when jc is known to be in the list. If x is 
the first term of the list, three comparisons are needed, one to determine whether the 
end of the I ist has been reached, one to compare x and the first term, and one outside the 
loop. If r is the second term of the list, two more comparisons are needed, so that a total 
of five comparisons are used. In general, if a is the /th term of the list, two comparisons 
will be used at each of the i steps of the loop, and one outside the loop, so that a total 
of 2/ + 5 comparisons are needed, Hence, the average number of comparisons used 
equals 

3 + 5 + 7 + * ■ ■ 4- (2n +1) 2{1 + 2 + 3 4- - ■ ■ + ») + /i 

n n 

In Section 3.2 we will show that 


1 4- 2 + 3 + ■ ■ ■ + n 


n(n -f- 1) 
2 


Hence, the average number of comparisons used by the linear search algorithm (when 
x is known to be in the list) is 

2 + FV21 

-— -h J = n i-2, 

n 


which is (Hn ) 

Remark: In this analysis it has been assumed that x is in the list being searched and 
it is equally likely that x is in any position. It is also possible to do an average-case 
analysis of this algorithm when x may not be in the list (see Exercise 13 at the end of 
this section). ■ 


Table 1 displays some common terminology used lo describe the time complexity 
of algorithms. For instance, an algorithm is said to have exponential complexity if it 
has time complexity 0(b n ), where b > 1, measured in terms of some specified type 
of operation. Similarly, an algorithm with time complexity 0(n b ) is said to have poly¬ 
nomial Complexity, The linear search algorithm has linear (worst- or average-case! 
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TABLE 1 Commonly Used Terminology for 
the Complexify of Algorithms. 

Complexity 

Terminology 

om 

Constant complexity 

Oflngn) 

Logarithmic complexity 

OOi 1 

Linear complexity 

0(n log ft) 

n log n complexity 

(){n p ) 

Polynomial complexity 

0(b"X where b > 1 

Exponential complexity 

0{n{) 

Factorial complexity 


complexify and the binary search algorithm has logarithmic (worst-ease; complexity, 
measured in terms of the number of comparisons used. 

A problem that is solvable using an algorithm with polynomial worst-case com¬ 
plexity is called tractable, since the expectation is that the algorithm will produce 
the solution to the problem for reasonably sized input in a relatively short time. How ¬ 
ever, if the polynomial in the big-0 estimate has high degree (such as degree 100 1 
or if the coefficients are extremely large, the algorithm may take an extremely long 
time to solve the problem. Consequently, that a problem can be solved using an algo¬ 
rithm with polynomial worst-case time complexity is no guarantee that the problem 
can be solved in a reasonable amount of time for even relatively small input values. 
Fortunately, in practice, the degree and coefficients of polynomials in such estimates 
are small. 

The situation is much worse for problems that cannot be solved using an algorithm 
with worst-case polynomial time complexity. Such problems are called intractable. 
Usually, but not always, an extremely large amount of time is required to solve the 
problem for the worst cases of even small input values, In practice, however, there tire 
situations where an algorithm with worst-case time complexity may be able to solve 
a problem much more quickly for most cases than for its worst case. When we are 
willing to allow' that some, perhaps small, number of cases may not be solved in a 
reasonable amount of time, the average-case time complexity is a better measure of 
how long an algorithm takes to solve a problem. Many problems important in industry 
are thought to be intractable but can be practically solved for essentially all sets of 
input that arise in daily life. Another way that intractable problems are handled when 
they arise in practical applications is that instead of looking for exact solutions of a 
problem, approximate solutions are sought. It may be the case that fast algorithms exist 
tor finding such approximate solutions, perhaps even with a guarantee that they do not 
differ by very much front an exact solution. 

Some problems even exist for which it can he shown that no algorithm exists for 
solving them. Such problems are called unsolvable (as opposed to solvable problems 
that can be. solved using an algorithm), The first proof that there are unsolvable problems 
was provided by the great English mathematician and computer scientist Alan Turing, 
The problem he showed unsolvable is the halting problem. This problem takes as 
its input a program together with input to this program. The problem asks whether 
the program will hall when executed with the input to the program. We will study the 
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halting problem in Section 3.1. (A biography of Alan Turing and a description of some 
of his other work can be found in Chapter 10.) 

The study of the complexity of algorithms goes far beyond what we can describe 
here. Note, however, that many solvable problems are believed to have the property 
that no algorithm with polynomial worst-ease time complexity solves them, but that 
once a solution is known, it can be checked in polynomial time. Problems for which a 
solution can be checked in polynomial time are said to belong to the class NP (tractable 
problems axe said to belong to class P). There is also an important class of problems, 
called NP-complete problems, with the property that if any of these problems can be 
solved by a polynomial worst-ease time algorithm, then all can be solved by polyno¬ 
mial worst-case time algorithms. Despite extensive research, no polynomial worst-case 
time algorithm has been found for any problem in this class. It is generally accepted, al¬ 
though not proven, that no NP-complete problem can be solved in polynomial time. For 
more information about the complexity of algorithms, consult the references, including 
ICoLeRi 90], for this section listed at the end of this book. 

Note that a big-0 estimate of the time complexity of an algorithm expresses how 
the time required to solve the problem changes as the input grows in size, in practice, 
the best estimate (that is, with the smallest reference function) that can be shown is 
used. However, big-0 estimates of time complexity cannot be directly translated into 
the actual amount of computer time used. One reason is that a big-0 estimate f (n) is 
Oig'JU'- where fin) is the time complexity of an algorithm and gin) is a reference func¬ 
tion, means that fin) r. Cg(ri) when n > k, where C and k are constants. So without 
knowing the constants C and k in the inequality, this estimate cannot be used to deter¬ 
mine an upper bound on the number of operations used. Moreover, as remarked before, 
the time required for an operation depends on the type of operation and the computer 
being used. (Also note that a big-0 estimate on the time complexity of an algorithm 
provides an upper, but not a lower, bound, on the worst-case time required for the ai- 
gorithm as a function of the input size. To provide a lower bound, a big-Theta estimate 
should be used. However, for simplicity, we will use big-0 estimates when describ¬ 
ing the time complexity of algorithms, with the understanding that big-Theta estimates 
would provide more information.) 

However, the lime required for an algortdim to solve a problem of a specified size 
can be determined if all operations can be reduced to the bit operations used by the 
computer. Table 2 displays the time needed to solve problems of various sizes with 


! TAB LI’ 2 The Computer Time Used by Algorithms. 




Problem Size 

Bit Operations Used 





ft 

1 log n 

n 

n log n 

rr 

2" 

n! 

10 

3 x 

10-® s 

3 X !0' s s 

10 7 s 

3CT 6 s 

3 x 10 4 s 

H! 3 

7 x 10 9 s 

10 7 s 

7 X i0‘ 7 s 

10 35 s 

4 X 10 13 yr 

* 

i(V 

1.0 X 10 S 5 

10" 6 s 

i x nr J s 

10" 3 s 

* 

* 

io 4 

1.3 x 10 8 s 

10 s s 

1 x |ir 4 s 

ifr' S 

* 

* 

I0» 

1.7 X 10“ 3 s 

HI 4 s 

2 X 10’ 3 s 

10 s 

* 

* 

10" 

2 x 10“ H s 

10 3 s 

2 x Hr- n 

17 min 

4 

4 
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an algorithm using the indicated number of bit operations. Times of more than lO 100 
years are indicated with an asterisk. (In Section 2.4 the number of bit operations used 
to add and multiply two integers will be discussed.) In the construction of this table, 
each bit operation is assumed to take 10“ 9 second, which is the time required for a bit 
operation using the fastest computers today. In the future, these times will decrease as 
faster computers are developed. 

It is important to know bow long a computer will need to solve a problem. For in¬ 
stance, if an algorithm requires )0 hours, it may be worthwhile to spend the computer 
time (and money ) required to solve this problem. But, if am algorithm requires 10 billion 
years to solve a problem, it would be unreasonable to use resources to implement this 
algorithm. One of the most interesting phenomena of modem technology is the tremen¬ 
dous increase in the speed and memory space of computers. Another important factor 
that decreases the time needed to solve problems on computers is parallel processing, 
which is the technique of performing sequences of operations simultaneously. Because 
of the increased speed of computation, increases in computer memory, and the use of 
algorithms that take advantage of parallel processing, problems that were considered 
impossible to solve 5 years ago are now routinely solved, and certainly 5 years from 
now this statement w'ill still be true. 


Exercises 

1. How many comparisons are used by the algorithm given 
in Exercise 10 of Section 2.1 to find the smallest natural 
number in a sequence of n natural numbers? 

1. Write the algorithm that puts the first four terms of a list 
of arbitrary' length in increasing order. Show that this al¬ 
gorithm has time complexity 0(11 in terms of the num¬ 
ber of comparisons used. 

3. Suppose that an element is known to be among the first 
four elements in a list of 32 elements. Would a lin¬ 
ear search or a binary search locate this element more 
rapidly? 

4. Determine the number of multiplications used to find x 2 
starting with .rand successively squaring (to find x 1 * * * * * 7 8 , x\ 
and so on), Is this a more efficient way to find jr* than by 
multiplying x by itself the appropriate number of times? 

5. Give a big-0 estimate for the number of comparisons 
used by the algorithm that determines the number of Is 
in a bit string by examining each bit of the string to de¬ 
termine whether it is a I bit (see Exercise 19 of Sec¬ 
tion 2.1). 

*6. a) Show that the following algorithm determines the 
number of 1 bits in the bit string $, 

procedure bitcounAS: bit string) 
count: - 0 

while 5^0 
begin 

arum : = couni + 1 
S : - 5 A (5 - 1 i 

end {count is the number of Is in 5} 


Here 5 - 1 is the bit string obtained by changing the 

rightmost i bit of S to a 0 and all the 0 bits to the right 

of this to Is. [Recall that 5 A (5 -- 1) is the bitwise 

AND of Aar,d 5 - 1.] 

b) How many bitwise AND operations are needed to 

find the number of 1 bits in a string S‘ } 

7. The conventional algorithm for evaluating a polynomial 
(Vr" + a„-ix”" 1 + ■ ■ ■ +■ ai* + a<j at x = c can be ex¬ 
pressed in pseudocode by 

procedure polynomiat(c, itu, a„: real 
numbers) 
power: - 1 
v: = do 

for i : = 1 to n 
begin 

power : =- power * r 
v : - y + a, * power 

end {y = a„c" - a n t c" 1 + • • ■ + me 4- £i (S } 

where the final value of y is the value of the polynomial 
at x = c. 

a) Evaluate 3.r + x + 1 at x - 2 by working through 
each step of the algorithm. 

b) Exactly how many multiplications and additions are 
used to evaluate a polynomial of degree n at x = c' 1 
(Do not count additions used to increment the loop 
variable.) 

8. There is a more efficient algorithm (in terms of the num¬ 
ber of multiplications and additions used) for evaluating 
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polynomials than the conventional algorithm de¬ 
scribed in the previous exercise. It is called 
Horner’s method. The following pseudocode 
shows how to use this method to lint! the value of 
a n x" + « fl i jf" ’ 1 + ■ ■ • + a i .r + Ac at a — c. 

procedure Homeric, a$, fij a ; . a„: real 

numbers) 
y : = tig 
for f: 1 to n 

y : = v * c + a„~, 

jv = a „ r n + fl n - 1 c ' , “ 1 + ■ ■■ + a t c + ct 0 } 

a) Evaluate 3.r + x + 1 at x = 2 by working through 
each step of the algorithm. 

b) Exactly how many multiplications and additions 
are used by this algorithm to evaluate a polynomial 
of degree ri at x = c ? (Do not count additions used 
to increment the loop variable.) 

d, How large a problem can be solved in 1 second using 
an algorithm di.it requires /fa) bit operations, where 
each bit operation is carried out in It) <J second, with 
the following values for /(«)? 

a) loan hi n c) nfoga 

d) n ; e) 2" f) n\ 

JO. How much time does an algorithm take to solve a prob¬ 
lem of size n if this algorithm uses 2n- + 2" bit oper¬ 
ations, each requiring 10 9 second, with the following 
values of ri '> 
at It) b) 20 

c) 50 d) 100 

11. How much time does an algorithm using 2* bit op¬ 
erations need if each bit operation takes the following 
amount of time? 

a) 10 * second 

b) ID' 1 second 

e) 10" 1: second 

12. Determine the least number of comparisons, or best- 
case performance, 

a) required to find the maximum of a sequence of n 
integers, using Algorithm 1 of Section 2.1. 

b) used to locate an element in a list of n terms with a 
linear search. 


c) used to locate an element in a list of n terms using 
a binary search. 

13. Analyze the average-case performance of the linear 
search algorithm, if exactly half the time element x is 
not in the list and if x is in the list it is equally likely to 
be in any position. 

14. An algorithm is called optimal for the solution of a 
problem with respect to a specified operation if there 
is no algorithm for solving this problem using fewer 
operations, 

a) Show that Algorithm 1 in Section 2.1 is an op¬ 
timal algorithm with respect to the number of 
comparisons of integers. {Note: Comparisons used 
for bookkeeping in the loop are not of concern 
here.) 

b) Is the linear search algorithm optimal with respect 
to the number of comparisons of integers (not in¬ 
cluding comparisons used for bookkeeping in the 
loop)? 

15. Describe the worst-case time complexity, measured in 
terms of comparisons, of the ternary' search algorithm 
described in Exercise 21 of Section 2.1. 

16. Describe the worst-case time complexity, measured 
in terms of comparisons, of the search algorithm de¬ 
scribed in Exercise 22 of Section 2.1. 

17. Analyze the worst-case time complexity of the algo¬ 
rithm you devised in Exercise 23 of Section 2.1 for lo¬ 
cating a mode in a list of nondeerfessing integers. 

18. Analyze the worst-case time complexity of the algo¬ 
rithm you devised in Exercise 24 of Section 2.1 for lo¬ 
cating all modes in a list of nondecreasing integers. 

19. Analyze the worst-case time complexity of the algo¬ 
rithm you devised in Exercise. 25 of Section 2.1 for 
finding the first term of a sequence of integers equal 
to some previous term. 

20. Analyze the worst-case time complexity of the algo¬ 
rithm you devised in Exercise 26 of Section 2.1 for 
finding all terms of a sequence that are greater than the 
sum of all previous terms. 

21. .Analyze the worst-case time complexity of the algo¬ 
rithm you devised in Exercise 27 of Section 2.1 for 
finding the first term of a sequence less than the im¬ 
mediately preceding term. 


.3 

he Integers and Division 

INTRODUCTION 

The part of discrete mathematics involving the integers and thetr properties belongs to 
the branch of mathematics called number theory. This section is the beginning of a 
three-section introduction to number theory. In this section we will review' some basic 
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EXAMPLE I 


concepts of number theory, including divisibility, greatest common divisors, and mod¬ 
ular arithmetic. In Section 2.4 we will describe several important algorithms from 
number theory, tying together the material in Sections 2.1 and 2.2 on algorithms and 
their complexity with the notions introduced in this section. For example, we will in¬ 
troduce algorithms for finding the greatest common divisor of two positive integers 
and for performing computer arithmetic using binary expansions. Finally, in Section 
2.5, we will continue our study of number theory by introducing some important re¬ 
sults and their applications to computer arithmetic and cryptology, the study of secret 
messages. 

The ideas that we will develop in this section are based on the notion of divisibility'. 
One important concept based on divisibility is that of a prime number. A prime is an 
integer greater than 1 that is divisible only by 1 and by itself. Determining whether 
an integer is prime is important in applications to cryptology. An important theorem 
from number theory, the Fundamental Theorem of Arithmetic, asserts that every pos¬ 
itive integer can be written uniquely as the product of prime numbers. Factoring in¬ 
tegers into their prime factors is important in cryptology. Division of an integer by a 
positive integer produces a quotient and a remainder. Working with these remainders 
leads to modular arithmetic, which is used throughout computer science. We will dis¬ 
cuss three applications of modular arithmetic in this section: generating pseudorandom 
numbers, assigning computer memory locations to files, and encrypting and decrypting 
messages. 


DIVISION 


When one integer is divided by a second, nonzero integer, the quotient may or may not 
be an integer. For example, 12/3 = 4 is an integer, whereas 11/4 = 2.75 is not. This 
leads to the following definition. 


DEFINITION 1. If a and b are integers with a ^ 0, we say that a divides b if 
there is an integer c such that b - ac. When a divides b we say that a is a factor of 
b and that b is a multiple of a . The notation a j b denotes that a divides b. We write 
a / b when a does not divide b. 


In Figure 1 a number line indicates which integers are divisible by the positive 
integer d. 


Determine whether 3 J 7 and whether 3 | 12. 

Solution: It follows that 3 /7, since 7/3 is not an integer. On the other hand, 3 | 12 
since 12/3 = 4. ■ 


--4-1-i-(--1-f- 

-itl -2d d 0 d 1d 34 

I’lCrli Rll 1 Integers Divisible by the Positive Integer d. 
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EXAMPLE 2 


THEOREM 1 


EXAMPLE 3 


THEOREM 2 


Let n and d be positive integers. How many positive integers not exceeding n are di¬ 
visible by dl 

Solution: The positive integers divisible by d are all the integers of the form dk, where 
k is a positive integer. Hence, the number of positive integers divisible by d that do 
not exceed n equals the number of integers k with 0 < dk n, or with 0 < k ^ nld. 
Therefore, there are [nid\ positive integers not exceeding n that are divisible by d. ■ 


Some of the basic properties of divisibility of integers are given in Theorem 1. 

Let a, t>, ‘to&I'chie tWdgiers. Tfcfiti 

... L-tff:6+ c); • 4 ' ' ■ 

3. lf<ij fraadfrj e,theaa|c. 

Proof: Suppose that a \ b and a \ c. Tlten, from the definition of divisibility, it follows 
that there are integers s and t with b = as and c = at. Hence, 

b + c - as + nr - «(,s + (). 

Therefore, a divides b +■ c. This establishes part (1) of the theorem. The proofs of parts 
(2) and (3) are left as exercises for the reader. □ 

PRIMES 


Every positive integer greater than 1 is divisible by at least two integers, since a positive 
integer is divisible by 1 and by itself. Integers that have exactly two different positive 
integer factors are called primes. 

2. A positive integer p greate than ! hesModpritnt if tbe only 
positive fiaHans i md p A positive integer th# kipeitter titan 1 sad is not 

pri me is called composite. 


The integer 7 is prime since its only positive factors are 3 and 7, whereas the integer 9 
is composite since it is divisible by 3. ■ 


The primes less than 100 are 2, 3. 5. 7, 11. 13, 17, 19, 23, 29, 31, 37, 41,43, 47. 
53. 59, 61. 67, 71, 73, 79,83, 89, and 97. 

The primes are the building blocks of positive integers, as the Fundamental Theo¬ 
rem of Arithmetic shows. The proof will be given in Section 3.2. 


THE FUNDAMENTAL THEOREM OF ARITHMETIC Evoy positive ateger 
can be written uniquely as the product of primes, where the prime fa sto r mre w ritte a 
in order of increasing size, (Here, a product can have zero, one, or mute than tm 
prime factor.) 
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EXAMPLE 4 


THEOREM 3 


EXAMPLE 5 


The nest example gives some prime factorizations of integers. 


The prime factorizations of 100, 641,990. and 1024 are given by 
100 = 2 ■ 2 ■ 5 ■ 5 - 2 2 5\ 

641 = 641, 


999 = 3-3-3-37 = 3 s ■ 37. 

1024 — 2-2-2-2-2'2’2’2-2-2 — 2 i0 . 


It is often important to show that a given integer is prime. For instance, in cryptol¬ 
ogy large primes are used in some methods for making messages secret. One procedure 
for showing that an integer is prime is based on the following observation. 


If rt is a composite integer, then n has a prime divisor less than or equal to Jn. 

Proof: If n is composite, it has a factor a with 1 < a < n. Hence, n = ab, where both 
a and b are positive integers greater than 1. We see that a " Jn or b < Jn. since 
otherwise ab > Jn ■ jn =■ n. Hence, n has a positive divisor not exceeding Jn. This 
divisor is either prime or. by the Fundamental Theorem of Arithmetic, has a prime 
divisor. In either case, n has a prime divisor less than or equal to Jn. □ 

From Theorem 3, it follows that an integer is prime if it is not divisible by any prime 
less than or equal to its square root. In the following example this observation is used 
to show that 101 is prime. 


Show that 101 is prime. 


Solution: The only primes not exceeding ,, MH are 2, 3. 5, and 7. Since 101 is not 
divisible by 2, 3,3, or 7 (the quotient of 101 and each of these integers is not an integer), 
it follows that 101 is prime. g 


Since every integer has a prime factorization, it would be useful to have a pro¬ 
cedure lor finding this prime lactorization. Consider the problem of finding the prime 
factorization of n. Begin by dividing n by successive primes, starting with the small¬ 
est prime, 2. If n has a prime factor, then by Theorem 3 a prime factor p not exceed¬ 
ing jn will be found. So, if no prime factor not exceeding Jn is found, then n is 
prime. Otherwise, if a prime factor p is found, continue by factoring nip. Note that 
nip has no prime factors less than p. Again, if nJp has no prime factor greater than 
or equal to p and not exceeding its square root, then it is prime. Otherwise, if it has 
a prime factor q, continue by factoring itlipq). This procedure is continued until the 
factorization has been reduced to a prime. This procedure is illustrated in the following 
example. 
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EXAMPLE 6 Find the prime factorization of 7007, 

Solution: To find the prime factorization of 7007. first perform divisions of 7007 by 
successive primes, beginning with 2. None of the primes 2, 3, and 5 divides 7007. How¬ 
ever, 7 divides 7007. with 7007/7 = 1001. Next, divide 1001 by successive primes, 
beginning with 1, it is immediately seen that 7 also divides 1001, since 1001/7 = 143. 
Continue by dividing 143 by successive primes, beginning with 7. Although 7 does not 
divide 143, 11 does divide 143, and 143/11 - 13. Since 13 is prime, the procedure is 
completed. It follows that the prime factorization of 7007 is 7 • 7 - 11 ■ 13 = 7“ ■ 11 ■ 13. 


Prime numbers were studied in ancient times for philosophical reasons. Today, 
web there are now highly practical reasons for their study. In particular, large primes play 
a crucial role in cryptography, as we will see in Section 2.5. It has long been known 
that there are infinitely many primes, a fact we will prove in Section 3-1. Since there 
are infinitely many primes, given any positive integer there are primes greater than this 
integer. There is an ongoing quest to discover larger and larger prime numbers; for al¬ 
most all the last 300 years, the largest prime known has been an integer of the special 
form 2^-1. where p is also prime. Such primes are called Mersenne primes, after 
the French monk Marin Mersenne. who studied them in the seventeenth century. The 
reason that the largest known prime has usually been a Mersenne prime is that there ts 
an extremely efficient test, known as the Lucas-Lehmer test, for determining whether 
2 P ] is prime. Furthermore, it is not currently possible to test numbers not of certain 
special forms anywhere near as quickly to determine whether they are prime. 


EXAMPLE 7 


The numbers l 2 - ! = 3, 2 ? - 1 = 7, and 2 5 - 1 - 31 are Mersenne primes, while 
2 1 f - 1 — 2047 is not a Mersenne prime since 2047 = 23 • 89. ■ 


Progress in finding Mersenne primes has been steady since computers were in¬ 
vented. As of late 1998, 37 different Mersenne primes were known, with six found 


Marin Mersenne (I588-164N). Mersenne was born in Maine. France, into a family of laborers and ai- 
Web “aided the College of Mans and the Jesuit College at La Flee he. He continued his education at the Sortwmnc, 
studying theology from 1609 to 1611. He joined the religious order of the Minims in 1611, a group whose 
name comes from the word mi'nmu (the members of this group considered themselves the least religious 
order). Besides prayer, the members of this group devoted their energy to scholarship and study, in 1612 he 
became a priest at the Place Royale in Paris; between 1614 and 1618 he taught philosophy at the Minim 
Convent at Severs. He returned to Paris in 1619, where his cell in the Minims de 1'Annnciade became 
a place for meetings of French scientists, philosophers, and mathematici.ans, including Fermat and Pas¬ 
cal, Mersenne corresponded extensively with scholars throughout Europe, serving as n clearinghouse for 
mathematical and scientific knowledge, a function later served by mathematical journals (and today also 
,lK ‘ Internet). Mersenne wrote books covering mechanics, mathematical physics, mathematics, music, 
and acoustics. He studied prime numbers and tried unsuccessfully to construct a formula representing all 
primes. In 164-4 Mersenne claimed that 2 P - 1 is prime for p - 2,3. 5. 7. 1 J, |7, 19 , J], 67, [27, 267 but 
is composite for all other primes less than 257. Il took over 309 years to determine that Mcr&enne’s claim 
was wrong five times. Specifically. 2 ! ‘ - 1 is not prime for p = 67 and p = 257 but is prune for p - 61, 
p - *7. and p - 107. it is also noteworthy that Mersenne defended two of the most famous men of hu 
tune, Descartes and Galileo, from religious critics. He also helped expose alchemists and astrologers as 

tTLiLldh. 
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EXAMPLES 


since 1990. The largest Mersenne prime known fas of late 1998) is 2’ 021777 — 1. a 
number with 909,526 digits. A communal effort, the Great Internet Mersenne Prime 
Search (GIMPS), has been organized to look for new' Mersenne primes. By the way. 
even the search for Mersenne primes has practical implications. One quality control 
test for supercomputers has been to replicate the Lucas-Lehmer test that establishes 
the primalily of a large Mersenne prime. 

Using trial division with Theorem 3 gives procedures for factoring and for primal- 
ity testing. However, these procedures are not efficient algorithms; many much more 
practical and efficient algorithms for these tasks have been developed. Factoring and 
primal ity testing have become important in the applications of number theory to cryp¬ 
tography. This has led to a great interest in developing efficient algorithms for both 
tasks. Clever procedures have been devised in the last 25 years for efficiently gener¬ 
ating large primes, However, even though powerful new factorization methods have 
been developed in the same time frame, factoring large numbers remains extraordi¬ 
narily more time consuming. Nevertheless, the challenge of factoring large numbers 
interests many people. There is a communal effort on the Internet to factor large num¬ 
bers, especially those of the special form k" n I, when: it is a small positive integer and 
n is a large positive integer (such numbers are called Cunningham numbers). At any- 
given time, there is a list of the '“Ten Most Wanted” large numbers of this type awaiting 
factorization. 


THE DIVISION ALGORITHM 


We have seen that an integer may or may not be divisible by another. However, when 
an integer is divided by a positive integer, there always is a quotient and a remainder, 
as the division algorithm shows. 


THE DIVISION ALGORI THM Let a be an integer and d a positive integer. Then 
there are unique integers q and r, with 0 r < d, such that a = dq + r. 


Remark: Theorem 4 is not really an algorithm, (Why not?) Nevertheless, we use its 
traditional name, 


DEFINITION 3. In the equality given in the division algorithm, d is called the 
divisor, a is called the dividend, q is called the quotient, and r is called the remainder. 


The following two examples illustrate the division algorithm. 


What are the quotient and remainder when 101 is divided by 11 ? 

Solution: We have 
101 - 11 9 + 2- 

Hence, the quotient when 101 is divided by 11 is 9, and the remainder is 2. 
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EXAMPLE 9 


EXAMPLE 10 


EXAMPLE 11 


What are the quotient and remainder when “11 is divided by 3? 

Solution: We have 
-11 = 31-4)+- 1. 

Hence, the quotient when - 11 is divided by 3 is -4, and the remainder is 1. 

Note that the remainder cannot be negative. Consequently, the remainder is not -2, 
even though 

HI = 3(—3) - 2, 

since r — -2 does not satisfy {) £ r < 3. ■ 


Note that the integer a is divisible by the integer d if and only if the remainder is 
zero when a is divided by d. 


GREATEST COMMON DIVISORS 
AND LEAST COMMON MULTIPLES 


The largest integer that divides both of two integers is called the greatest common 
divisor of these integers. 


DEFINITION 4. Let a and i be integers, not both zero. The largest integer d such 
that d j a and d\bv& called the greatest common divisor of a and h. The greatest 
common divisor of a and b is denoted by gedfo, b). 


The greatest common divisor of two integers, not both zero, exists because the set 
of common divisors of these integers is finite. One way to find the greatest common 
divisor of two integers is to find all the positive common divisors of both integers and 
then take the largest divisor. This is done in the following examples. Later, a more 
efficient method of finding greatest common divisors will be given, 


What is the greatest common divisor of 24 and 36? 

Solution: The positive common divisors of 24 and 36 are 1, 2, 3. 4, 6, and 12. Hence, 
gcd(24, 36) = 12. ■ 


What is the greatest common divisor of 17 and 22? 

Solution: The integers 17 and 22 have no positive common divisors other than 1, so 
that gcd(17, 22) = 1 ■ 


Since it is often important to specify that two integers have no common positive 
di visor other than 1, we have the following definition. 
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EXAMPLE 12 


EXAMPLE 13 


EXAMPLE 14 


DEFINITION 5, The integers a and b are relatively prime if their greatest com¬ 
mon divisor is 1. 


From Example 11 it follows that the integers 17 and 22 are relatively pnme, since 
gcd(17, 22) = 1. ■ 


Since we often need to specify that no two integers in a set of integers have a 
common positive divisor greater than !, we make the following definition. 

DEFINITION 6. The integers <ij, a 2 .. a n are pairwise relatively prime if 

gedfflo dj) = 1 whenever 1 < i < j < n. 


Determine whether the integers 10, 17, and 21 are pairwise relatively prime and 
whether the integers 10, 19, and 24 are pairwise relatively prime. 

Solution: Since gcdflO, 17) ^ J.gcdflO. 21) = 1, and gcdf!7, 21) ~ 1, wc conclude 
that 10, 17, and 21 are pairwise relatively prime. 

Since gcd(l0,24.) — 2 > I, we see that 10, 19, and 24 are not pairwise relatively 
prime. ■ 


Another way to find the greatest common divisor of two integers is to use the prime 
factorizations of these integers. Suppose that the prime factorizations of the integers a 
and b t neither equal to zero, are 

* = p'l'pV ■•'!%. ft = Pi' Pi "■/£. 

where each exponent is a nonnegative integer, and where all primes occurring in the 
prime factorization of either a or b arc included in both factorizations, with zero expo¬ 
nents if necessary. Then sedfa, b) is given by 

gcdfa b) - p'r Ui '" bi ■ • ■ p-; 

where minfx y) represents the minimum tif the two numbers ,v and y. To show that 
this formula for gcdi'u, b ) is valid, we must show that the integer on the right-hand side 
divides both a and b. and that no larger integer also does. This integer does divide both 
a and b. sasce the power of each prime in the factorization does not exceed the power 
of this prime in either the factorization of a or that of b. Further, no larger integer can 
divide both a and h, because the exponents of the primes in this factorization cannot be 
increased, and no other primes can be included. 


Since the prime faciorizahonx of 120 and 500 are 120 2 3 ■ 3 ■ 5 and 500 = 2 2 ■ 5 : ‘. 

the greatest common divisor is 

gcd(120,500) - 2 ml[,,3,1, 3 mirrfl - (r> 5 milin ; ' : ■= 2 2 3 l> 5' ^ 20, ■ 
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EXAMPLE 15 


THEOREM $ 


Prime factorizations can also be used to find the least common multiple of two 
integers. 


DEFINITION 7. The least common multiple of the positive integers a and h is 
die smallest positive integer that is divisible by both a and b. The least common 
multiple of a and b is denoted by lctnfa, b). 


The least common multiple exists because the set of integers divisible by both a and 
b is nonempty, and every nonempty set of positive integers has a least element (by the 
well-ordering property, which will be discussed in Chapter 3). Suppose that the prime 
factorizations of a and b are as before. Then the least common multiple of a and h is 
given by 

kma.b) = p 7 “ : 

where maxLr, >) denotes the maximum of the two numbers jt and y. This formula is 
valid since a common multiple of a and b has at least max(n„ b,) factors of p ( in its 
prime factorization, and the least common multiple has no other prime factors besides 
those in a and b. 


What is the least common multiple of 2 3 3 5 7* and 2 4 3 J ? 
Solution: We have 

Icm(2-’3 5 7 2 ,2 4 3 3 ) - 2" M, <-A4J3oiiiMS.3>7aiMi2,n» _ 2 i $ s j 2 


The following theorem gives the relationship between the greatest common divisor 
and least common multiple of two integers, ft can be proved using the formulae we 
have derived for these quantities. The proof of this theorem is left as an exercise for the 
reader. 

Let# and b be positive integers. Then 
nb = gcd(a, b) ■ lctn(a, b). 

MODULAR ARITHMETIC 

In some situations we care only about the remainder of an integer when it is divided 
by some specified positive integer. For instance, when we ask what time it will be (on 
a 24-hour clock) 50 hours from now, we care only about the remainder when 50 plus 
the current hour is divided by 24. Since we are often interested only in remainders, we 
have special notations for them. 

DEFINITION 8. Let a be an integer and in be a positive integer. We denote by a 
mod m the remainder when a is divided by m. 
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It follows from the definition of remainder that a mod m is the integer r such that 
a - qm ■f r and 0 < r < in. 


EXAMPLE 16 We see that 17 mod 5 = 2 f -133 mod 9 - 2, and 2001 mod 101 =82. ■ 


We also have a notation to indicate that two integers have the same remainder when 
they are divided by the positive integer m. 


DEFINITION 9. If a and b are integers and m is a positive integer, (hen a is 
congruent to b modulo m if m divides a - b. We use the notation a™ b (mod m) to 
indicate that a is congruent to b modulo m. If a and b are not congr uent modulo tn, 
we write a # b (mod m). 


Note that o = b (mod m) if and only if a mod m - h mod m 


EXAMPLE 17 Determine whether 17 is congruent to 5 modulo 6 and whether 24 and 14 tire congruent 

modulo 6. 

Solution: Since 6 divides 17 - 5 =■ 12, we see that 17=5 (mod 6). However, since 
24 - 14 = 10 is not divisible by 6, we see that 24 f 14 (mod 6). B 


Tiie great German mathematician Karl Friedrich Gauss developed the concept of 
congruences at the end of the eighteenth century'. 

The notion of congruences has played an important role in the development of num¬ 
ber theory. The following theorem provides a useful way to work with congruences. 


Karl Friedrich Gauss (1777-1855 J. Karl Friedrich Gaum, the son of a bricklayer, was a child prodigy, 
Web Hc demonstrated his potential at the age: of 10, when he quickly solved a problem assigned by a teacher In 
keep the class busy, Tire teacher asked the students to find the sum of the first 100 positive integers. Gauss 

realized that this sum could be found by forming 50 pairs, each with the sum 101:1 -t-100,2+W.50+51. 

This brilliance attracted the sponsor ship of patrons, including Duke Ferdinand of Brunswick, who made it 
possible for Gauss to attend Caroline College and the University of Gottingen. While a student, he invented 
the method of least squares, which is used to estimate the most likely value of a variable from experimental 
results. In 1796Gauss made a fumlameiilat discovery in geometry, advancing a subject that had not advanced 
since ancient times. He showed that a 17-sided regular polygon could be drawn using just a ruler and 
compass, 

tn 1799 Gauss presented the first rigorous proof of the Fundamental Theorem of Arithmetic, which 
states that a polynomial of degree n has exactly n roots (counting multiplicities!. Gauss achieved world¬ 
wide fame when he successfully calculated the orbit of the first asteroid discovered, Ceres, using scanty 
data. 

Gauss was called the Prince of Mathematics by his contemporary mathematicians Although Gauss 
is noted for his many discoveries in geometry, algebra, analysis, astronomy, and physics, he had a special 
interest in number theory, which can be seen from his statement '‘Mathematics is the queen of the sciences, 
and the theory 1 of numbers is the queen of mathematics." Gauss laid the foundations for modem number 
theory with the publication of his book Disquixirwus Arithmetirtw in I SOI. 
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THEOREM 6 


THEOREM 7 


EXAMPLE IS 


EXAMPLE 19 


Let m be a positive integer. The integers aand b are congruent modulo m if and only 
if there is an integef k such that a = b + fare. 

Proof: i {a = b (mod m), then m \ la - b). This means that there is an integer k such 
that £? h ~ km, so that a = b + km. Conversely, if there is an integer k such that 
a ~ h + bn, then km = a - b. Hence, m divides a - b, so that a = b (mod m). □ 

The following theorem shows how congruences work with respect to addition and mul¬ 
tiplication. 


Let m be a positive integer. If a = b (mod m) and c & d (mod m), then 
a + c ** b + d (mod m) and ac^ bd. (mod *). 

Proof: Since a = b (mod m) and r = d (mod m), there are integers s and f with b = 
a + srn and d ~ c + tm. Hence, 

h - d = (a + .tm} + (<■ - tm) — (a + r) + m{s + t) 

and 

bd = [a -t .'«t i;c + tm) — as + m(at + cs + stm). 

Hence, 

a + r 3 b + d (mod mi and ac = bd (mod m). □ 


Since 7 — .2 (mod 5) and 11^1 nuod 5), it follows from Theorem 7 that 
18 = 7+11 = 2*] =3 (mud 5) 
and that 

77 -- 7 - 11 = 2 * I - 2 (mod 51 . ■ 

APPLICATIONS OF CONGRUENCES 

Number theory has applications to a wide range of areas. We will introduce three appli¬ 
cations m this section: the use of congruences to assign memory locations to computer 
files, the generation of pseudorandom numbers, and cryptosystems based on modular 
arithmetic. 


Hashing Functions The central computer al your school maintains records for each 
student. How can memory locations be assigned so that student records can be retrieved 
quickly? The solution to this problem is to use a suitably chosen hashing function. 
Records are identified using a key, which uniquely identities each student's records. 
For instance, student records are often identified using the Social Security number of 
the student as the key, A hashing function h assigns memory' location h(k) to the record 
that has k as its key, 

fn practice, many dif ferent hashing functions are used. One of the most common is 
the function 
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h(k) - k mod m 

where m is the number of available memory locations. 

Hashing functions should be easily evaluated so that files can be quickly located. 
The hashing function h(k) - k mod m meets this requirement; to find h(k), we need 
only compute the remainder when k is divided by m. Furthermore, the hashing function 
should be onto, so that all memory locations arc possible. The function h(k) - tmod m 
also satisfies this property. 

For example, when m - Ill, the record of the student with Social Security number 
06421 2848 is assigned to memory location 14, since 

/a 0642 1.848 1 = 064212848 mod 111 = 14. 

Similarly, since 

*(037149212) = 037149212 mod 11! - 65, 

the record of the student with Social Security number 037149212 is assigned to memory 
location 65. 

Since a hashing function is not one-to-one (since there are more possible keys than 
memory locations), more than one file may be assigned to a memory location. When 
this happens, we say that a collision occurs. One way to resolve a collision is to assign 
the first free location following the occupied memory' location assigned by the hashing 
function. For example, after making the two earlier assignments, we assign location 15 
to the record of the student with the Social Security number 107405723. To see this, 
first note that h(k) maps this Social Security number to location 14, since 

hi 107405723) = 107405723 mod 11 1 = 14, 

but this location is already occupied (by the file of the student with Social Security- 
number 064212848), However, memory location 15, the first location following mem¬ 
ory location 14, is free. 

There are many more sophisticated ways to resolve collisions that are more efficient 
than the simple method we have described. These are discussed in the references on 
hashing functions given at the end of the book. ■ 


KXAMI’f f 20 Pseudorandom Numbers Randomly chosen numbers are often needed for computer 

simulations. Different methods have been devised for generating numbers that have 
properties of randomly chosen numbers. Because numbers generated by systematic 
methods are not truly random, they are called pseudorandom numbers. 

The most commonly used procedure for generating pseudorandom numbers is the 
linear congruential method. We choose four integers: the modulus m, multiplier 
a, increment c, and seed jc a . with 2 s: a < m.O ^ c < m, and 0 ^ x u < m. We 
generate a sequence of pseudorandom numbers {a,,}, with 0 < x„ < m for till n, by 
successively using the congruence 

.v,;. i = (cix„ + <:) mod m 

(This is an example of a recursive definition, discussed in Section 3.3. In that section 
we will show that such sequences arc well defined.) 

Many computer experiments require the generation of pseudorandom numbers be¬ 
tween 0 and i To generate such numbers, we divide numbers generated with a linear 
congruential generator by the modulus: that is, wc use the numbers jc„fm, 
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EXAMPLE 21 


For instance, the sequence of pseudorandom numbers generated by choosing m - 
9, a - 7, c ~ 4, and Xg = 3, can be found as follows: 


a'i — 7+o + 4 = 7 ■ 3 
x 2 = + 4 = 7 ■ 7 

+3 = 1x2 +4 = 7-8 
A'4 = 7 T ] + 4 — 7'6 
Aj = 7 jc 4 +4=7-1 
•tf, — 7 a' 5 + 4 = 7-2 
x~i = 7A(i + 4 = 7-0 
.\'s = 7+7 +4 — 7*4 
A g = + 4 = 7 ■ 5 


+ 4 = 25 mod 9 = 7, 
+ 4 =53 mod 9 = 8 , 
+ 4 = 60 mod 9 = 6, 
+ 4 = 46 mod 9=1, 
+ 4 = 11 mod 9 = 2. 
+ 4 = 18 mod 9 = 0, 
+ 4 = 4 mod 9 = 4, 
+ 4 = 32 mod 9 = 5, 
-r 4 = 39 mod 9 = 3. 


Since ,tg = xg and since each term depends only on the previous term, the following 
sequence is generated: 


3, 7,8, 6,1, 2, 0,4, 5,3. 7, 8,6, 1,2, 0, 4, 5, 3, . 


This sequence contains nine different numbers before repeating. 

Most computers do use linear congruential generators to generate pseudorandom 
numbers. Often, a linear congruential generator with increment c = 0 is used. Such a 
generator :s called a pure multiplicative generator. For example, the pure multiplicative 
generator with modulus 2 31 - 1 and multiplier 7 s - 16,807 is widely used. With these 
values, it can be shown that 2 31 - 2 numbers are generated before repetition begins, ■ 


CRYPTOLOGY 

Congruences have many applications to discrete mathematics and computer science. 
Discussions of these applications can be found in the suggested readings given at the 
end of the book. One of the most important applications of congruences involves cryp¬ 
tology, which is the study of secret messages. One of the earliest known uses of cryptol¬ 
ogy was by Julius Caesar. He made messages secret by shifting each letter three letters 
forward in the alphabet (sending the last three letters of the alphabet to the first three). 
For instance, using this scheme the letter B is sent to E and the letter X is sent to A. This 
is an example of encryption, that is. the process of making a message secret. 

To express Caesar's encryption process mathematically, first replace each letter by 
an integer from 0 to 25, based on its position in the alphabet. For example, replace A 
by 0, K by 10, and Z by 25. Caesar’s encryption method can be represented by the 
function/ that assigns to the nonnegative integer25. the integer f(p) in the set 
(0,1,2.25f with 

f(p) = iP + 3) mod 26. 

In the encrypted version of the message, the letter represented hyp is replaced with the 
letter represented by ( p + 3) mod 26. 


What is the secret message produced from the message “MEET YOU IN THE PARK" 
using the Caesar cipher? 

Solution: First replace the letters in the message with numbers. This produces 
12 4 4 19 24 14 20 8 13 19 7 4 15 0 17 10, 
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Now replace each of these numbers p by f(p) = (p + 3) mod 26, This gives 
15 7 7 22 1 17 23 11 16 22 10 7 18 32013. 

Translating this back to letters produces the encrypted message “PHHW BRX LQ 
WKH SDUN.” ■ 


To recover the original message from a secret message encrypted by the Caesar 
cipher, the function the inverse off. is used. Note that the function f~ ] sends an 

integer/* from {0,1,2.25} to/' 1 !/*) = (p- 3) mod 26. In other words, to find the 

original message, each letter is shifted back three letters in the alphabet, with the first 
three letters sent to the last three letters of the alphabet. The process of determining the 
original message from the encrypted message is called decryption. 

There are various ways to generalize the Caesar cipher. For example, instead of 
shifting each letter by 3, we can shift each letter by k. so that 

fip) = ip + k) mod 26, 

Such a cipher is called a s hi ft cipher. Note that decryption can be carried out using 
/' ‘(p) = ip - k) mod 26. 

Obviously, Caesar’s method and shift ciphers do not provide a high level of security. 
There are various ways to enhance this method. One approach that slightly enhances 
the security is to use a function of the form 

f{p) = iap + b ) mod 26, 

where a and b are integers, chosen such that/ is a bijection. (Such a mapping is called 
an affine transformation.) This provides a number of possible encryption systems. The 
use of one of these systems is illustrated in the following example. 


EXAMPLE 22 What letter replaces the letter K when the function fip) = {Ip + 3) mod 26 is used 

for encryption? 

Solution : First, note that 10 represents K. Then, using the encryption function speci¬ 
fied, it follows that /(10) = (7 ■ 10 + 3) niod 26 = 21. Since 21 represents V. K is 
replaced by V in the encrypted message. ■ 


Caesar’s encryption method, and the generalization of this method, proceed by re¬ 
placing each letter of the alphabet by another letter in the alphabet. Encryption methods 
of this kind are vulnerable to attacks based on the frequency of occurrence of letters in 
the message. More sophisticated encryption methods are based on replacing blocks of 
letters with other blocks of letters. There are a number of techniques based on modular 
arithmetic fox encrypting blocks of letters. A discussion of these can be found in the 
suggested readings listed at the end of the book. 


Exercises 

1. [toes 17 divide each of the following numbers? 2. Show that if a is an integer other than 0, then 

a) 6K b) K4 c) 357 d) 1001 a) 1 divides a. b) a divides 0. 
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.1 Show that part (2) of Theorem 1 is true. 

4. Show that part (31 of Theorem 1 is true, 

5. Show that if a j b and b\ a, where a and b are integers, 
then a = b or a ~ -b. 


6. Show that if it, h, r, and d are integers such that a j c 
and h ■ d, then ab \ at , 

7. Show that if a, b, and c are integers such that ac : be, 
then a b 

ft. Are the following integers primes'' 
it) 19 b) 27 el 93 

d) 101 e) 107 f) 113 


9, In each of the following cases, what are the quotient 
and remainder? 

a) 19 is divided hv 7 b) - 111 is divided by 11 

e) 789 is divided by 23 d) 1001 is divided by 13 

e) 1) is divided by 19 f) 3 is divided by 5 

g) -1 is divided by 3 h) 4 is divided by 1 

10 . Find the prime factorization of each of the following, 

a) 39 b) 81 c) 101 

d) 143 e) 289 f) 899 

11. Find the prime factorization of 10!. 

*12. How many zeros are there at the end of 10017 
*13. An irrational number is a re a! number _r that cannot 
be written as the ratio of two integers. Show that kg : 3 
is an irrational number. 

14. Which positive integers less than 12 arc relatively 
prime to 12? 

15. [Determine whether the following sets of integers arc 
pairwise relatively prime. 

a) (11. 15. 19) b) (14.15,21) 

c) (12, 17.31,37) d) (7, 8,9, 11) 

16. We call a positive integer perfect if it equals the sum 
of its positive divisors other than itself. 

a> Show that 6 and 28 are perfect. 

b) Show that 2‘"" \2 1 ’ - 1) is a perfect number when 
2’’ - 1 is prime, 

17. I •el m be a positive integer. Show that a ~ b (mod m) 
if a mod m - b mod m, 

18. Let m be a positive integer. Show that a mod m = b 
mod mifu = b (mod m) 

19. Show that if 2™ - 1 is prime, then n is prime. \Hint: Use 

the identity 2 “' - 1 = (2 s -l)-(2 4f *' + + 

2" + l).|" 


20. Determine whether each of the following integers is 
prime, verifying some of Mersenne’s claims, 
a) 2 ; - I b) 2 ! - I 
c) 2" - 1 d) 2' 5 - 1 


21. The value of the Kuier ^-function at the positive in¬ 
teger n is de lined to be the number of positive integers 
less than or equal to n that are relatively prime to n, 
[Note: 4 is the Greek letter phi.) Find 

a) 4(4). b) 4(10). c) 4(13). 

22. Show that n is prime i!' and oniy if 4>{n) = n - 1. 

23. What is the value of <b{p*) when p is prime and k is a 
positive integer'? 


24. What are the greatest common divisors of the following 
pairs of integers'? 

a) 2 2 • 3’’ ■ 5 s , 2 s • 3 3 - 5 3 

b) 2-3-5- 7- 11 ■ 13,2“ -3 s - N ■ I7 !4 

c) 17, 17 17 

d) 13 

e) 0, 5 

f) 2 ■ 3 - 5 ■ 7,2 ’ 3 ■ 5 ■ 7 

*25. Show' that if n and k are positive integers, then \n!k] = 
l(n - f)/fj + i, 

26. Show that if a is an integer and d is a positive inte¬ 
ger greater than I, then the quotient and remainder ob¬ 
tained when a is divided by d are [a)d\ and a — d[ald\, 
respectively. 

27. Find a formula for the integer with smallest absolute 
value that is congruent to an integer a mod til o m, where 
m is a positive integer. 

28. Evaluate the following quantities 

a) -17 mod 2 b) 144 mod 7 

c) -101 mod 13 d) 199 mod 19 

29. Evaluate the following quantities. 

a) 13 mod 3 b) —97 mod i 1 
c) 155 mod 19 d> -221 mod 23 
31). List live integers that are congruent to 4 modulo 12. 

31. Decide whether each of the following integers is con¬ 
gruent to 5 modulo i 7. 

a) 80 b) 103 c) -29 d) -122 

32. If the product of two integers is 2 7 3 8 5 2 7 u and their 
greatest common divisor is 2 3 3 4 5, what is their least 
common multiple? 

33. Show that if a and b are positive integers then ab = 
gcdffl, !>) 1cm («, b). [Him: Use the prime factorizations 
of a and b and the formulae for gcd{a, b) and lcm(a, b) 
in terras of these factorizations. ] 

34. Show that if a = b (mod m) and e = d (mod m), where 
a. h, l, d, and m are integers with m e: 2, then a- c = 
b ~ d (mod a;). 

35. Show that if n | m, where n and m arc positive integers 

greater than i, and if a b (mod m), where a and b arc 

integers, then a = h (mod n), 

36. Show that if a, b, c, and m arc integers such (hat m ' 2 a 2, 
c > 0, ;md a = h (mod m), then ae = be (mod me), 

37. Show that ae = be (mod m), where a, b, c, and m are 
integers with m se 2, does not necessarily imply that 
a = h (mod m), 

38. Show- 1 that if a, b, and m are integers such that m a 2 
and b (mod m), then ged(n, mi = ged(f>, m), 

39. Show (hat if a, b, k. and m are integers such that k £ 

1. m - 2, and a - h (modm), then a 1 = /^(mod m) 
whenever k is a positive integer. 

40. Which memory locations are assigned by the hashing 
function h(k) = k mod 301 to the records of students 
with the following Social Security numbers? 

a) 104578690 b) 432222187 
c) 372201919 d) 501338753 
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41 . A parking lot has 31 visitor spaces, numbered from 0 to 
30. Visitors are assigned parking spaces using the hash¬ 
ing function h(k) = k mod 31, where k is the number 
formed from the firs! three digits on a visitor’s license 
plate. 

a) Which spaces are assigned by the hashing function 
to cars that have the following first three digits on 
their license plates? 

317,918.007, 100.111,310 

b) Describe a procedure visitors should follow to find 
a free parking space, when the space they are as¬ 
signed is occupied. 

42. What sequence of pseudorandom numbers is gener¬ 
ated using the linear congruential generator Jt„ +1 = 
(4 1 „ + 1) mod 7 with seed ,t ti = 3? 

43. What sequence of pseudorandom numbers is gener¬ 
ated using the pure multiplicative generator ,r„ +1 = 
3,t„ mod 11 with seed xa = 2? 

44 . Write an algorithm in pseudocode for generating a se¬ 
quence of pseudorandom numbers using a linear con¬ 
gruential generator. 

45. Encrypt the message “DO NOT PASS GO” by trans¬ 
lating the letters into numbers, applying the encryption 
function given, and then translating the numbers back 
into letters. 

a) fip) - (p + 3) mod 26 (the Caesar cipher) 

b) f(,p) = (p + 13) mod 26 
C) j'(p) = (3p + 7) mod 26 

46. Decrypt the following messages encrypted using the 
Caesar cipher, 

a) FOXH MHDQV 

b) WHVWWRGDB 
C) HDWGLPVXP 

Books are identified by an International Standard 
Book Number (ISBN), a 10-digit code xtxj ... tjo, 
assigned by the publisher. These 10 digits consist of blocks 
identifying the language, the publisher, the number as¬ 
signed to the book by its publishing company, and finally, 


a 1-digit check digit that is either a digit or the letter 
X (used to represent 10). This check digit is selected 
so that V*=i ixj - 0 (mod II) and is used to detect er¬ 
rors in individual digits and transposition of dig¬ 
its, 

47. The first nine digits of the ISBN of the third edition of 
this book are 0-07-053965. What is the check digit for 
this book? 

48. The ISBN of Elementary Number Theory and Its Ap¬ 
plications, 3d ed,, is 0-201 -57(789-1. where Q is a 
digit. Find the value of £). 

49. Determine whether the check digit of the ISBN for this 
textbook was computed correctly by the publisher. 

50. Find the smallest positive integer with exactly n differ¬ 
ent factors when n is 

a) 3. b) 4. c) 5, d) 6. e) 10, 

51. Can you find a formula or rule for the nth term of a se¬ 
quence related to the prime numbers or prime factor¬ 
izations so that the initial terms of the sequence have 
the following values? 

a) 0, 1,1,0,1,0,1,0, 0,0, 1.0, 1, ... 

b) 1,2.3, 2,5, 2,7.2, 3, 2, 11, 2, 13, 2, ... 

c) 1,2,2, 3, 2,4, 2.4, 3,4, 2, 6,2,4, ... 

d) 1, 1, CO, 1, 1, 1,0,0,1, 1,0, M, ... 

e) 1,2,3. 3,5,5, 7, 7, 7,7, 11, 11, 13, 13, 

I) 1, 2, 6, 30, 210, 2310, 30030. 510510, 9699690, 
223092870, ... 

52. Can you find a formula or rule for the nth term of a se¬ 
quence related to the prime numbers or prime factor¬ 
izations so that the initial terms of the sequence have 
the following values? 

a) 2,2, 3,5,5, 7,7, 11, 11, II, 11, 13, 13, ... 

b) 0, 1,2, 2, 3, 3, 4, 4, 4,4, 5, 5, 6, 6, ... 

c) 1.0,0, 1,0. 1,0, I, I. 1,0, 1,0, 1, ... 

d) 1, -1, -1, 0. -1, 1, -I. O, 0, I, -1, 0, -1. 

1 , 1 , ... 

e) 1, 1. 1,1, 1,0, 1, I, 1,0, 1,0, 1,0.0, ... 

0 4, 9, 25, 49, 111. 169, 289, 361, 529, 841, 961, 
1369. ... 


2.4 _ 

Integers and Algorithms 

INTRODUCTION 


As mentioned in Section 2.1, the terra algorithm originally referred to procedures 
for performing arithmetic operations using the decimal representations of integers. 
These algorithms, adapted for use with (unary representations, are the basis for 
computer arithmetic. They provide good illustrations of the concept of an algorithm 
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and the complexity of algorithms. For these reasons, they will be discussed in this 
section. 

There are many important algorithms involving integers besides those used in arith¬ 
metic. We will begin our discussion of integers and algorithms with the Euclidean al¬ 
gorithm. It is one of the most useful algorithms, and perhaps the oldest algorithm in 
mathematics. We will also describe an algorithm for finding the base b expansion of a 
positive integer for any base b. 


THE EUCLIDEAN ALGORITHM 


The method described in Section 2.3 for computing the greatest common divisor of 
web two integers, using the prime factorizations of these integers, is inefficient. The rea¬ 
son is that it is time-consuming to find prime factorizations. We will give a more 
efficient method of finding the greatest common divisor, called the Euclidean algo¬ 
rithm. This algorithm has been known since ancient, times. It is named after the an¬ 
cient Greek mathematician Euclid, w ho included a description of this algorithm in hrs 
Elements. 

Before describing the Euclidean algorithm, we will show' how it is used to find 
gcd(9l, 287). First, divide 287, the larger of the two integers, by 91, the smaller, to 
obtain 

287 = 91 ■ 3 + 14. 

Any divisor of 91 and 287 must also be a divisor of 287 -91-3 = 14. Also, any divisor 
of 91 and 14 must also be a divisor of 287 - 91-3+ 14. Hence, the greatest common 
divisor of 91 and 287 is the same as the greatest common divisor of 91 and 14. This 
means that the problem of finding ged(91,287) has been reduced to the problem of 
Finding gcd(91,14). 

Next, divide 91 by 14 to obtain 

91 = 14-6 + 7. 

Since any common divisor of 91 and 14 also divides 91 - 14*6 = 7 and any common 
divisor of 14 and 7 divides 91, it follows that gcd(91,14) = gedf J4, 7). 

Continue by dividing 14 by 7, to obtain 

14 = 7 2. 

Since 7 divides 14, it follows that gcd( 14,7) = 7, and since gcd(287, 91) - gcd(91,14) 
gcd(14, 7) = 7, the original problem has been solved. 

We now describe how the Euclidean algorithm works in generality. We will use 
successive divisions to reduce the problem of finding the greatest common divisor of 
two positive integers to the same problem with smaller integers, until one of the integers 
is zero, 

The Euclidean algorithm is based on the following result about greatest common 
divisors and the division algorithm. 


Euclid | c. 3S0 B-C.E, }. Euclid was the author of the most successful mathematics book ever written, the 
Web which appeared in over 1000 different editions From ancient to modern times. Little is known 

about Euclid's life, other than that he taught at the famous academy at Alexandria. Apparently, Estdid did 
not stress applications When a student asked what he would get by teaming geometry, Euclid explained 
that knowledge was worth acquiring for its own sake and told his servant to give the student a com "since 
he must make a profit from what he learns," 
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LEMMA 1 


EXAMPLE 1 


Let a = bq + r, where a, b, q, and r are integers. Then gcd(u, b) = ged(fc f r). 

Proof: If we can show that the common divisors of a and b are the same as the common 
divisors of b and r, we will have shown that, gcdfa, b) = gcd{b, rj, since both pairs must 
have the same greatest common divisor. 

So suppose that d divides both a and b. Then it follows that d also divides a-bq = r 
(from Theorem 1 of Section 23), Hence, any common divisor of a and b is also a 
common divisor of h and r. 

Likewise, suppose that d divides both b and r, Then d also divides bq + r = a. 
Hence, any common divisor of b and r is also a common divisor of a and b. 

Consequently, gcd(«, b) - ged{6, r). □ 

Suppose that a and b are positive integers with a - ’ b. Let r ( ; = a and - b. 
When we successively apply the division algorithm, we obtain 

ri) = + n t) s n < r\, 

r i - r?q 2 + 0 Sfj < r?, 

r» -2 = r„-iq a . i + r„ 0 <=' r„ < r„-,. 

r n -i = r n q„. 

Eventually a remainder of zero occurs in this sequence of successive divisions, since 
the sequence of remainders a - r 0 > r } > r 2 > - ■ ■ s 0 cannot contain more than a 
terms. Purthenuore, it follows from Lemma 1 that 

gcd(a, b) = gcdfro.r,) = gcd(n, r 2 ) = ■■■ = gcd(r„_ 2 , r „.., ) 

- FC'Ur, t..j = gcd(r„, 0) = 

Hence, the greatest common divisor is the last nonzero remainder in (he sequence of 
divisions. 


Find the greatest common divisor of 414 and 662 using the Euclidean algorithm. 
Solution: Successive uses of the division algorithm give: 

662 = 414 • 1 + 248 
414-248-1^166 
248 =166-1+82 
166 = 82-2 + 2 
82 = 2- 41. 

Hence, gcd(414, 662) = 2, since 2 is the last nonzero remainder. ■ 

The Euclidean algorithm is expressed in pseudocode in Algorithm I. 
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THEOREM 1 


I algorithm i The Euclidean Algorithm. 

! procedure gcd(& b: positive integers) 

x i - « 

| y: = b 

! while v # 0 

| begin 

| r ■ — x mod v 

; .v :■= v 

i v ; =■ r 

end {gcd(«, b) is a} 


(n Algorithm l, the initial values of a and y are a and b, respectively. At each stage of the 
procedure,.t is replaced by y, and y is replaced by x mod y. which is the remainder when 
v is divided by y. This process is repeated as long as y # (T The algorithm terminates 
when v = 0, and the value of t at that point, the last nonzero remainder in the procedure, 
is the greatest common divisor of a and b. 

We will study the time complexity of the Euclidean algorithm in Section 3.3, where 
we will show that the number of divisions required to find the greatest common divisor 
of o and b, where a s b, is 0(log b\ 


REPRESENTATIONS OF INTEGERS 


In everyday life we use decimal notation to express integers. For example, 965 is 
used to denote 9 ■ lt> + 6 ■ 10 + 5 . However, it is often convenient to use bases other 
than 10. In particular, computers usually use binary' notation (with 2 as the base) 
when carrying out arithmetic, and octal (base 8} or hexadecimal (base 16) notation 
when expressing characters, such as letters or digits. In fact, we can use any posi¬ 
tive integer greater than 1 as the base when expressing integers. This is stated in the 
following theorem. 


Let b be a positive integer greater than 1. Then if n is a positive integer, it can be 
expressed uniquely in the form 

n = a k b k + a k ,ib k ~ l + -- - +a|b + 

where k is a nonnegative integer, oq, fl[...., a k are nonnegative integers less than b, 
and a k # 0. 


The proof of this theorem can be found in the suggested readings referred to at the end 
of the book. The representation of n given in Theorem 1 is called the base b expansion 
of n. The base b expansion of n is denoted by (a t: a k . i ■ ■■ a\an)t,. For instance, (245)* 
represents 2 • -t- 4 • 8 + 5 - 165. 

Choosing 2 as the base gives binary expansions of integers, In binary notation 
each digit is either a 0 or a I. In other words, the binary' expansion of an integer is 
just a bit string. Binary' expansions (and related expansions that are variants of binary 
expansions) are used by computers to represent and do arithmetic with integers. 
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EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


What is the decimal expansion of the integer that has f 101011111); as its binary expan¬ 
sion? 

Solution: We have 

(101011111)2 = 2 8 + 2 6 + 2 + + 2 3 + 2 2 + 2 4 1 = 351. B 


Sixteen is another base used in computer science. The base 16 expansion of an 
integer is called its hexadecimal expansion. Sixteen different digits are required for 
such expansions. Usually, the hexadecimal digits used are 0,1, 2, 3,4, 5, 6, 7, 8,9, A, 
B. C, D. E, and F, where the letters A through F represent the digits corresponding to 
the numbers 10 through 15 (in decimal notation). 


What is the decimal expansion of the hexadeci mal expansion of (2AE0B)jg? 
Solution: We have 

(2AE0B) [(, = 2 ■ 16 4 4 10 ■ 16 ? 4 14 ■ I6 2 + 0-16+11 = (I7J627) ]0 . 


Since a hexadecimal digit is represented using four bits, bytes, which are hit strings of 
length eight, can be represented by two hexadecimal digits. For instance, (1 LlQOIOl); 
= (£5 )i 6 since (1110); = (E) 16 and (0101), = (5} iS . 

We will now describe an algorithm for constructing the base h expansion of an 
integer n. First, divide n by b to obtain a quotient and remainder, that is, 

n = bq o + a 0 , 0 s' fly < b. 

The remainder, «o, is the rightmost digit in the base h expansion of n. Next, divide q<) 
by b to obtain 

qo = bqi * a |, 0 < a\ < b. 

We see that <+ is the second digit from the right in the base b expansion of n. Continue 
this process, successively dividing the quotients by b, obtaining additional base h digits 
as the remainders. This process terminates when we obtain a quotient equal to zero. 


Find the base 8 expansion of (12345 he. 

Solution: First, divide 12345 by 8 to obtain 
12345 = 8-1543 + 1, 

Successively dividing quotients by 8 gives 
1543 - 8- 192 + 7 
192 = 8*24 + 0 
24 = 8 ■ 3 + 0 
3=8-0 + 3. 
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1.12 


Since the remainders tire the digits of the base 8 expansion of 12345, it follows that 
(12345 ) r a - (30()71) s . ■ 


The pseudocode given in Algorithm 2 finds the base b expansion {a k . ] ■ ■ ■ a 
of the integer n. 


Ai.CtmtTHM 1 Constructing base h expansions. 

procedure base b expanxinn(iy. positive integer) 
q : = n 
j k : = 0 

I while q ^ 0 

begin 

i a k : = q mod b 

b ~ L#t ! 

I k:= k - I 

end (the base b expansion of n is (a k [ ■ • ■ rqfluJj} 


In Algorithm 2, q represents the quotient obtained by successive divisions by b, starting 
with q = n. Hie digits in the base b expansion are the remainders of these divisions 
and are given by q mod b. The algorithm terminates when a quotient q = 0 is reached. 

ALGORITHMS FOR INTEGER OPERATIONS 

The algorithms for performing operations with integers using their binary expansions 
are extremely important in computer arithmetic. We will describe algorithms for the 
addition and the multiplication of two integers expressed in binary notation. We will also 
analyze ihe computational complexity of these algorithms, in terms of the actual number 
oi bit operations used. 1 hroughout this discussion, suppose that the binary' expansions 
o f a and b are 

a - t«„ -ia n . tqufria, b - (Ivt b„~: - 1 • b| Vli- 

so that a and b each have n bits [putting bits equal to 0 at the beginning of one of these 
expansions if necessary). 

Consider the problem of adding two integers in binary notation. A procedure to per¬ 
form addition can be based on the usual method for adding numbers with pencil and paper. 
This method proceeds by adding pairs of binary digits together with carries, when they 
occur, to compute the sum of two integers. This procedure will now be specified in detail. 
To add a and A first add their right mu si lists This gives 

«ll + f’l! - I’i.i ■ 2 r x (J . 

where .q : . is the rightmost bit in the binary expansion of a -r b and e® is the carry, which 
is either 0 or I. Then add the next pair nf bits and the carry. 

ft | ' b' + f[) - f| ■ 2 t X; , 

where y is ihe next bit (from the right) in the binary expansion of a * h, and c- is the 
carry. ( onti nne (his process, adding the corresponding bits in the two binary expansions 
and the carry, to determine the next bit tram the right in the binary expansion of hi -t b. 
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At the last stage, add a„i. b„ ,, and c „-2 to obtain q-i - 2 + s„-\. The leading bit of the 
sum is y„ - c„ ]. This procedure produces the binary expansion of the sum, namely, 
(I t b = (A'sS.h-[Sh-2'‘'>Mq)2- 


EXAMPLE 5 


I! 10 
10 11 
11(1111 

FMiliRK ! 


Adding 
( 1111 th and 


Adda = (lllOh and b = (101 lh. 

Solution: Following the procedure specified in the algorithm, first note thai 
) -t- h o — 0 ■ l -■ 0-2 - 1, 
so that Cfi = 0 and .vq = 1. Then, since 
ti[ + h\ + Co — 1 * 1 + 0 — l 1 2 + 0, 
it follows that c t = 1 and iq = 0. Continuing, 
o 2 t b 2 -t C| = 1 1 0 i 1 = i • 2 4 0 , 
so that ci = I and s 2 = 0. Finally, since 

+ by + ci _ 1+1 + 1 - 1 - 2 * 1 , 

it follows that C 3 - 1 and 53 = 1. This means that ,v 4 = cj - 1. Therefore, S - 
a -r h ~ (11001 ) 2 . This addition is displayed in Figure 1. ■ 

The algorithm for addition can be described using pseudocode as follows. 


\I+;OR]TH\I 3 Addition of Integers, 

procedure add(a. b: positive integers) 

{the binary expansions of a and b are ■ - ■ fpoij); 

and (b n -[b n -2 • ■ ■ bth + respectively} 

c : = 0 

for j : = 0 to n - ] 

begin 

d = . (aj ■+ bj ~ C.-V2J 
s,j := cij + bj + c - 2d 
c ; = d 

end 

s n c 

{the binary expansion of the sum is (,v<+ . 1 - - - .( 0 ) 3 } 


Next, the number of additions of bits used by Algorithm 3 will be analyzed. 


EXAMPLE 6 How many additions of bits are required to use Algorithm 3 to add two integers with n 

bits (or less) in their binary representations'’ 

Solution: Two integers are added by successively adding pairs of bits and, when it 
occurs, a carry. Adding each pair of bits and the carry requires three or fewer additions of 
bits. Thus, the total number of additions of bits used is less than three times the number 
of bits in (he expansion. Hence, the number of additions of bits used by Algorithm 3 to 
add two /i-hit integers is Oi n ). R 
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Next, consider the multiplication of two n-bit integers a and b. The conventional 
algorithm (used when multiplying with pencil and paper) works as follows. Using the 
distributive law, we see that 

n -1 «-i 

ab = (iVii ,V = Nb.2 
h u ; = 0 

We can compute ab using this equation. We first note that abj - aifb, - 1 and ab ; = 
0 if h, = 0. Each time we multiply a term by 2, we shift its binary expansion one place 
to the left and add a zero at the tail end of the expansion. Consequently, we can obtain 
{ab >2 by shifting the binary expansion of ab, j places to the left, adding j zero bits 
at the tail end of this binary expansion. Finally, we obtain ah by adding the n integers 
abpj, j = 0, 1,2. n-1. 

The following example illustrates the use of this algorithm. 


EXAMPLE 7 Find the product of a = (110b and b = (TOl h. 

Solution: First note that 



ah {l 

■2° 

= (110)2 

■ 1 ■ 2 n 

= (HO);, 


ab\ 

■ 2 1 

= (110) 2 

■ 0 • 2 1 

= (0000)2, 

1 1 0 

and 





1 0 1 

l 1 f) 

abi • 

■ 2 2 

- (1 10 >2 

■ 1 - 2 2 

= (11000) 2 . 


!) 0 (S 
1 I 0 
11110 
FIGURE 2 
Multiplying 
(110); and 
(101);. 


To find the product, add (110 b, (0000)2, and (11 OflO )j. Carrying out these additions (us¬ 
ing Algorithm 3, including initial zero bits when necessary) shows that ab = (111 10F. 
This multiplication is displayed in Figure 2. ■ 


This procedure for multiplication can be described using pseudocode as follows. 


ALGORITHM 4 Multiplying Integers, 

procedure mulripiyla. b: positive integers) 

{the binary expansions of a and b are (<?„_, a n _ 2 ■ * ■ a^atfe 
and (b„ \b n j ■ ■ ■ M’nF. respectively! 
for j : = 0 to u - 1 

begin 

if b, = 1 then c, : = a shifted j places 
else Cj : = 0 
end 

— t' n -: are the partial products] 

0 

for j : = 0 to n - 1 

P- = P-* c y 
{p is the value of ah} 
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Next, we determine the number of additions of bits and shifts of bits used by Al¬ 
gorithm 4 to multiply two integers. 


EXAMPLE 8 How many additions of bits and shifts of bits are used to multiply a and h using Algo¬ 

rithm 4? 

Solution: Algorithm 4 computes the products of a and b by adding the partial products 
tit, C], — and c„ [.When bj - 1, we compute the partial product c; by shifting the 
binary' expansion of aj bits. When bj = 0, no shifts arc required since c, = 0. Hence, 
to find all n of the integers ab f 2 ! , j — 0,1. n - 1, requires at most 

O + I+ I + '^ + ji-I 

shifts. Hence, by Example 4 in Section 1.8 the number of shifts required is 0(n 2 ). 

To add the integers ab. from j = 0 to j = n - l requires the addition of an 
n-bit integer, an (n + l)-bit integer,..., and a (2n}-bit integer. We know- from Example 
8 that each of these additions requires O(n) additions of bits. Consequently, a total of 
Oin 1 ) additions of bits are required for all rt additions. ■ 


Surprisingly, there are more efficient algorithms than the conventional algorithm 
for multiplying integers. One such algorithm, which uses 0(n l5S5 ) bit operations to 
multiply re-bit numbers, will be described in Chapter 5, 


Exercises 

1. Use the Euclidean algorithm to find 

a) sed(I2, 18). b) gcdflll, 201). 

c) gcdtlOOl, 1331). di gcd{ 12345, 54321). 

2. Use the Euclidean algorithm to find 

a) gedf 1,5) b) gcddOO, 101) 

e) gcd( 123.277) d) gedf1529, 14039) 
e) golf 1529, 14038) f) gcd(11111, 111111) 

3. How many divisions are required to find gcd(2l, 34) 
using the Euclidean algorithm? 

4. How many divisions arc required to find gcc!(34, 55) 
using the Euclidean algorithm? 

5. Convert the following integers from decimal notation 
to binary notation. 

a) 231 b) 4532 c) 97644 

t*. Convert the following integers from decimal notation 
to binary notation. 

a) 321 b) 1023 c) 100632 

7. Convert the following integers from binary notation to 
decimal notation, 
a) I 111) 
bl 10 0000 0001 
e) I 0101 0101 

d) 110 1001 0001 0000 


8, Convert the following integers from binary notation to 
decimal notation. 

a) 1 1011 b) 10 Kill 0101 

o ii ion mo d) in iioooooi ini 

9. Devise a simple method for converting from hexadec¬ 
imal notation to binary notation. 

10. Devise a simple method for converting from binary no¬ 
tation to hexadecimal notation, 

11. Convert each of the following integers from hexadeci¬ 
mal notation to binary' notation. 

a) ROE h) 135AB 

e) ABBA d) DEFACED 

12. Convert each of the following integers from binary' no¬ 
tation to hexadecimal notation, 

a) nil 0111 b) 1010 1010 I0JD 

e) 111 Dili 0111 0111 

13. Show that every positive integer can be represented 
uniquely as the sum of distinct powers of 2. (Hint: Con¬ 
sider binary' expansions of integers.) 

14. It can be shown that every integer can be uniquely rep¬ 
resented in the form 

*13* ' t’t-13‘ : 1 + ■■■ -f C|3 + t'n. 
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where f . = -1,0, or 1 for j = 0, 1,2,_ k. Expan¬ 

sions of tins type tire called balanced ternary expan¬ 
sions, Find the balanced ternary expansions of 
a)? b) 13 c) 37 d) 79 

15. Show that a positive integer is divisible by 3 if and only 
if the sum of its decimal digits is divisible by 3. 

16. Show that a positive integer is divisible by 11 if and 
only if (he difference of the sum of its decimal digits 
in even-numbered positions and the sum of its decimal 
digits in odd-numbered positions is divisible by II, 

17. Show that a positive integer is divisible by 3 if and only 
if the difference of the sum of its binary digits in even- 
numbered positions and the sum of its binary digits in 
odd-numbered positions is divisible by 3. 

One’s ciimplement representations of integers are used 
to simplify computer arithmetic. To represent positive and 
negative integers with absolute value less than 2 n a to¬ 
tal of n bits is used. The leftmost bit is used to represent 
[he sign. A 0 bit in this position is used for positive inte¬ 
gers. and a 1 bit in this position is used for negative in¬ 
tegers. For positive integers the remaining bus tire- identi¬ 
cal to the binary expansion Of the integer. For negative in¬ 
tegers, the remaining hits are obtained by first finding the 
binary expansion of the absolute value of the integer, and 
then taking the complement of each of these bits, where 
the complement of a 1 is a 0 and the complement of a 0 
is a 1. 

IK. Find the one's complement representations, using bit 
strings of length six, of the following integers. 
a) 22 b) 31 c) -7 d) -19' 

19, What integer does each of the following one's comple¬ 
ment representations of length five represent? 

a) ironi bi onoi c) iooo i d* uni 

20, It n f is a positive integer less than 2' l_1 , how is the one's 
complement representation of ■ m obtained from the 
one’s complement of m. when bit stnngs of length n 
are used' 5 

21, How is the one’s complement representation of the sum 
of two integers obtained from the one’s complement 
representations of these integers? 

22, How is the one’s complement representation of the dif¬ 
ference of two integers obtained from the one’s com¬ 
plement representations of these integers? 

23, Show that the integer m with one’s complement rep¬ 
resentation ; ■ ■ ■ ti|on'} can be found using the 

equation m — cj^ . 11 2' 1 1 - li - 

Two’s complement representations of integers are also 
used to simplify computer arithmetic and are used more 
commonly lhan one’s complement representations. To rep¬ 
resent an integer a with -2" 1 < .t £ 2"" 1 - ! for a spec¬ 
ified positive integer n, a total of n bits ts used. The leftmost 
bn is used lo represent the sign. A 0 bit in this position is 
used for positive integers, and a I bit in this position is used 


for negative integers, just as in one’s complement expan¬ 
sions For a positive integer, the remaining bits arc identi¬ 
cal to the binary expansion of the integer. For n negative 
integer, the remaining bits are the bits of the binary ex¬ 
pansion of 2" 1 - |.v|. Two’s complement expansions of in¬ 
tegers are often used by computers because addition and 
subtraction of integers can be performed easily using these 
expansions, where these integers can be either positive or 
negative. 

24. Answer Exercise 18. but this time find the two’s com¬ 
plement expansion using bit stnngs nf length si x, 

25. Answer Exercise 19 if each expansion is a tw o's com¬ 
plement expansion of length five. 

26 . Answer Exercise 20 for two’s complement expansions. 

27. Answer Exercise 21 for two’s complement expansions, 

28 . Answer Exercise 22 for two’s complement expansions. 

29. Show that the integer m with two’s complement rep¬ 
resentation (lz b _,c ;„_2 ■ ■ ■ £f[c!o) can be found using the 
equation m = 

31). Give a simple algorithm for forming the two's comple¬ 
ment representation of an integer front its one’s com¬ 
plement representation. 

31. Sometimes integers are encoded by using four-digit bi¬ 
nary expansions to represent each decimal digit. This 
produces the binary coded decimal form of the in¬ 
teger. For instance, 791 is encoded in this way bv 
01 i 11 Oft 10001. How many bits are required to repre¬ 
sent a number with n decimal digits using this type of 
encoding’ 1 

A Cantor expansion is a sunt of the form 

a„nl a n f(« - 1 }! + ■■■ -r aj2! * «i I b 

where a, is an integer with 0 is a. s f for i - 1, 

2. n. 

32. Find the Cantor expansions of 

a) 2 b) 7 c) 19 

di 57 e) 1000 f) 1,000,000 

*33. Describe an algorithm that finds the Cantor expansion 
of an integer. 

*34, Describe an algorithm to add two integers from their 
Cantor expansions. 

35. Add (1011 D- and (11010)2 by working through each 
step of the algorithm for addition given in the text. 

36. Multiply (11 iil;> and (lOIOJj by working through each 
step of the algorithm for multiplication given in the 
lext. 

37. Describe an algorithm for finding the difference of two 
hinary expansions. 

38. Estimate the number of bit operations used to subtract 
two binary' expansions. 

39. Devise an algorithm that, given the binary expansions 
of the integers a and b, determines whether a > b, a = 
b, or a < b. 
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40. How many bit operations does the comparison algo¬ 
rithm from Exercise 39 use when the larger of a and 
b has n bits in its binary expansion ? 


41. Estimate the complexity of Algorithm 2 for finding the 
base b expansion of an integer n in terms of tire number 
of divisions used. 


25 __ 

Applications of Number Theory 

INTRODUCTION 

Number theory has many applications, especially to computer science. In Section 2.3 
we described several of these applications, including hashing functions, the genera¬ 
tion of pseudorandom numbers, and shift ciphers. This section continues our intro¬ 
duction to number theory, developing some key results and presenting two important 
applications: a method for performing arithmetic with large integers and a recently in¬ 
vented type of cryptosystem, called a public key system. In such a cryptosystem, we do 
not have to keep encryption keys secret, since knowledge of an encryption key does not 
help someone decrypt messages in a realistic amount of time. Privately held decryption 
keys arc used to decrypt messages. 

Before developing these applications, we wall introduce some key results that play 
a central role in number theory and its applications. For example, we will show how 
to solve systems of linear congruences modulo pairwise relatively prime integers using 
the Chinese Remainder Theorem, and then show how to use this result as a basis for 
performing arithmetic with large integers. We will introduce Fermat’s Little Theorem 
and the concept of a pseudoprime and will show how to use these concepts to develop 
a public key cryptosystem. 


SOME USEFUL RESULTS 


An important result we will use throughout this section is that the greatest common 
divisor of two integers a and b can be expressed in the form 

set + lb, 

where s and l are integers. In other words, ged{a b) can be expressed as a linear 
combination with integer coefficients of a and b. For example, gcd(6, 14) ^ 2, and 
2 = (- 2) ■ 6 + 1 - 14. We state this fact as Theorem 1. 


THEOREM l ff a and b are positive integers, then there exist integers s and t such that gedfa, b) = 

sa + tb. 


We will not give a formal proof of Theorem i, but we will provide an example of a 
method for finding a linear combination of two integers equal to their greatest common 
divisor. (In this .section, we will assume that a linear combination has integer coeffi¬ 
cients.) The method proceeds by working backward through the divisions of the Eu¬ 
clidean algorithm. 
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EXAMPLE 1 


LEMMA 1 


Express gcd(252.198) = 18 as a linear combination of 252 and 198. 

Solution; To show that gcd{252, 198) = 18, the Euclidean algorithm uses the follow¬ 
ing divisions: 

252 = 1-198 + 54 

198 = 3 ■ 54 + 36 

54 = 1 ■ 36 r 18 

36 = 2 • 18, 

Using the next-to-Iast division (the third division), we can express gcd(252, 198) = 18 
as a linear combination of 54 and 36. We find that 


18 - 54 - 1 -36. 


The second division tells us that 
36 - 198 - 3 ■ 54, 

Substituting this expression for 36 into the previous equation, we can express 18 as a 
linear combination of 54 and 198. We have 

18 = 54 - 1 ■ 36 - 54 - 1 ■ (198 - 3 ■ 54) = 4 > 54 - 1 * 198. 

The first division tells us that 

54 - 252 - 1 - 198. 

Substituting this expression for 54 into the previous equation, we can express 18 as a 
linear combination of 252 and 198. We conclude that 

18 = 4- (252 - 1 • 198) - 1 • 198 = 4 ■ 252 - 5 ■ 198, 

completing the solution. ■ 


The method used in Example 1 works for any pair of positive integers. [There are 
more efficient methods for expressing gcd(a, b ) as a linear combination of a and b\ 
consult the references at the end of the book to learn more about these methods.] 

We will use Theorem 1 to develop several useful results. One of our goals will be 
to prove the part of the Fundamental Theorem of Arithmetic asserting (hat a positive 
integer has at most one prime factorization. We will show that, if a positive integer has 
a factorization into primes, where the primes are written in nondeercasing order, then 
this factorization is unique. 

First, we need to develop some results about divisibility. 


if a, b, and c are positive integers such that gedfq, b) ~ 1 and a | be, then a \ c. 

Proof: Since gctlfct, b) = I, bv Theorem 1 there are integers s and 1 such that 
sa + tb = ]. 

Multiplying both sides of this equation by c, we obtain 
sac +- the ~ c. 
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LEMMA 2 


EXAMPLE 2 


THEOREM 2 


Using Theorem 1 of Section 2.3, we can use this last equation to show that a ! c. By 
part 2 of that theorem, a j the. Since a \ sac and a \ the, by part 1 of that theorem, we 
conclude that a divides sac + the, and hence a \ c. This finishes the proof. □ 

We will use the follow ing generalization of Lemma 1 in the proof of uniqueness of 
prime factorizations, (The proof of Lemma 2 is left as an exercise in Section 3.2, since 
it can be most easily carried out using the method of mathematical induction, which 
will be covered in that section.) 

If P is a prime and p j a^ai where each is an kM&r, tea p E $ fcr 
some i. 

We can now show that a factorization of an integer into primes is unique. That is, 
we will show' that every integer can be written as the product of primes in nondecreasing 
order in at most one way. This is part of the Fundamental Theorem of Arithmetic. We 
will prove the other part, that every" integer has a factorization into primes, in Sec¬ 
tion 3.2. 

Proof (of the uniqueness of the prime factorization of a positive integer): Suppose 
that the positive integer n can be written as the product of primes in two different 
ways, say, n = p\pi ■ ■■ p : , and n = (pep- ■ ■■ q h each p, and q, are primes such that 
Pi s ^ ^ ■■■ <p t and<j, 5ft < ■■■ < q { . 

When w : e remove all common primes from the two factorizations, we have 

Pi: P‘2 * ’ 1 P‘ii — Qk Qh.'' * Qjo’ 

where no prime occurs on both sides of this equation and it and v are positive integers. 
By Lemma 2 it follows that p, t divides q,. for some k. Since no prime divides another 
prime, this is impossible. Consequently, there can be at most one factorization of n into 
primes in nondecreasing order, □ 

Lemma I can also be used to prove a result about dividing both sides of a con¬ 
gruence by the same integer. We have shown (Theorem 7 in Section 2.3) that we can 
multiply both sides of a congruence by the same integer. However, dividing both sides 
of a congruence by an integer does not always produce a valid congruence, as the fol¬ 
lowing example show's 


The congruence 14 =» 8 (mud 6) holds, but both sides of this congruence cannot be 
divided by 2 since 14/2 ^ 7 and 8/2 - 4, but 7 f 4 (mod 6). ■ 


However, using Lemma 1, we can show that we can divide both sides of a congru¬ 
ence by an integer relatively prime to the modulus. This is stated as Theorem 2. 

Let m be a positive integer and let a, fi, and <:• be integers. If ac * be (mod m) and 
gcd(c, m) = 1. then a = b (mod m). 

Proof: Sinccac s fie (mod m), m \ ac he - c(a-b). By Lemma I , since gcd(e:, im) -- 
1, it follows that m\a- b. We conclude that it - h (mod m). O 
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LINEAR CONGRUENCES 


A congruence of the form 
ax - h {mod m) 

where m is a positive integer, a and b are integers, and x is a variable, is called a linear 
congruence. Such congruences arise throughout number theory and its applications. 

How can we solve the linear congruence ax = b (mod m), that is, find all integers 
x that satisfy this congruence? One method that we will describe uses an integer a such 
that aa = 1 {mod m), if such an integer exists. Such an integer a is said to be an inverse 
of a modulo m. Theorem 3 guarantees that an inverse of a modulo m exists whenever 
a and m are relatively prime. 



THEOREM 3 


of a modulo m is congruent to a modulo m. ) 

Proof: By Theorem 1, since gcd(a, m) — 1, there are integers s and t such that 
sa + tm = 1. 

This implies that 

sa trra-l ( mod m). 

Since tm -- CM mod m), it follows that 
sa = 1 (mod m). 

Consequently, s is an inverse of a modulo m. That this inverse is unique modulo m is 
left as Exercise 9 at the end of this section. □ 

The proof of Theorem 3 describes a method for finding the inverse of a modulo m 
when a and m are relatively prime: find a linear combination of a and m that equals 1 
(which can be done by working backward through the steps of the Euclidean algorithm); 
the coefficient of a in this linear combination is an inverse of a modulo m. We illustrate 
this procedure in Example 3. 


EXAMPLE 3 


Find an inverse of 3 modulo 7. 


Solution: Since gcd(3, 7} = 1, Theorem 2 tells us that an inverse of 3 modulo 7 exists. 
The Euclidean algorithm ends quickly when used to find the greatest common divisor 
of 3 and 7: 

7 = 2-3 + 1. 

From this equation we see that 
-2-3 + 1 -7 = 1. 

This shows that -2 is an inverse of 3 modulo 7. (Note that every integer congruent to 
-2 modulo 7 is also an inverse of 3, such as 5, -9, 12, and so on.) ■ 
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EXAMPLE 4 


W’&> 


EXAMPLE 5 


When we have an inverse a of a modulo tn, we can easily solve the congruence 
ax = b (mod m) by multiplying both sides of the linear congruence by 3, as Example 
4 illustrates. 


What are the solutions of the linear congruence 3.r = 4 (mod 7) 7 

Solution: By Example 3 we know that ~2 is an inverse of 3 modulo 7. Multiplying 
both sides of the congruence by -2 shows that 

-2 ■ 3jt = -2 ■ 4 (mod 7). 

Since -6 = 1 (mod 7) and -8 = 6 (mod 7), it follows that if x is a solution, then 
x = -8 = 6 {mod 7). 

We need to determine whether ever)' x with x = 6 (mod 7) is a solution. Assume 
that x = 6 (mod 7). Then, by Theorem 7 of Section 2.3, it follows that 

3-v = 3-6 = 18 = 4 (mod 7), 


u'hieh shows that all such x satisfy the congruence. We conclude that the solutions to 
the congruence are the integers x such that ,v « 6 (mod 7), namely, 6, 13, 20,... and 

-1,-8, -15. ■ 

THE CHINESE REMAINDER THEOREM 

Systems of linear congruences arise in many contexts. For example, as we will see 
later, they are the basis for a method that can be used to perform arithmetic with large 
integers. Such systems can even be found as word puzzles in the writings of ancient 
Chinese and Hindu mathematicians, such as that given in Example 5. 


In the first century, the Chinese mathematician Sun-Tsu asked: 

There are certain things whose number is unknown. When divided by 3, the re¬ 
mainder is 2; when divided by 5, the remainder is 3; and when divided by 7, the 
remainder is 2. What will be the number of things? 

This puzzle can be translated into the following question: W'hat are the solutions of 
the systems of congruences 

x = 2 (mod 3), 

x •• 3 (mod 5), 

x - 2 (mod 7)? 

We will solve this system, and with it Sun-Tsu’s puzzle, later in this section. ■ 


The Chinese Remainder Theorem, named after the Chinese heritage of problems 
involving systems of linear congruences, states that when the moduli of a system of 
linear congruences are pairwise relatively prime, there is a unique solution of the system 
modulo the product of the moduli. 
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THEOREM 4 


EXAMPLE 6 


THE CHINESE REMAINDER THEOREM Let m h l*ff*iro!eb- 

tively prime positive integers. The system 

x » aj (mod »i), 

x = a2 (mod ms), ^ , ; 


.t a a „ (mod m„) 

has a unique solution modulo m = m]m 2 (That is, flSis© k *mimOmx with 

0 £ x < m, and all other solutions are congruent madtiki m © this tohitfe®.} 

Proof: To establish this theorem, we need to show that a solution exists and that it is 
unique modulo m. We will show that a solution exists by describing a way to construct 
this solution: showing that the solution is unique modulo m is Exercise 20 at the end of 
this section. 

To construct a simultaneous solution, first let 
M k = mlm k 

for k - 1,2,,... n. That is, M k is the product ofthe moduli except for m*. Since m, and 
rn k have no common factors greater than 1 when / ^ ic, it follows that gcdi'mj. M k ) - 1. 
Consequently, by Theorem 3, we know' that there is ait integer y k , an inverse of M k 
modulo nth, such that 

M k y k = 1 (mod rat). 

To construct a simultaneous solution, form the sum 
x ~ a\M iyi + a 2 M 2 yi ■+■■■+ a„M n y n . 

We will now show that x is a simultaneous solution. First, note that since Mj = 
0 (mod m> I whenever j ^ k, all terms except the Lth term in this sum are congru¬ 
ent to 0 modulo mj. Since M k y k — 1 (mod m k ) we see that 

x = a k M k y t = a k (mod m k ), 

for k = 1,2. n. We have shown that x is a simultaneous solution to the n congru¬ 
ences. O 

The following example illustrates how to use the construction given in the proof of 
Theorem 4 to solve a system of congruences. W r e will solve the system given in Example 
5, arising in Sun-Tsu’s puzzle. 


To solve the system of congruences in Example 5, Erst let m - 3-5-7 = 105, Mi = 
m/3 = 35, M 2 = m/5 = 21, and M k = mf 7 = 15, We see that 2 is an inverse of M| = 
35 modulo 3, since 35 = 2 (mod 3); 1 is an inverse of = 21 modulo 5, since 21 = 
1 {mod 5); and 1 is an inverse of M 3 : 15 (mod 7), since 15 - 1 (mod 7). The solutions 
to this system are those x such that 

x - it] Mi>’] + UiMisy 4- a k M-i = 2 - 35 *2 + 3- 21-1 + 2-15-1 (mod 105) 
= 233 - 23 (mod 105). 
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EXAMPLE 7 


EXAMPLE 8 


It follows that 23 is the smallest positive integer that is a simultaneous solution. We 
conclude that 23 is the smallest positive integer that leaves a remainder of 2 when 
divided by 3, a remainder of 3 when divided by 5, and a remainder of 2 when divided 
by 7. ' ■ 


COMPUTER ARITHMETIC WITH LARGE INTEGERS 


Suppose that m 2 , are pairwise relatively prime integers greater than or equal 

to 2 and let m be their product. By the Chinese Remainder Theorem, we can show (see 
Exercise 18) that an integer a with 0£a< m can be uniquely represented by the n- 

tuple consisting of its remainders upon division by m;,i ~ 1,2. n. That is, we can 

uniquely represent a by 

(a mod mi, a mod nh, a mod m n ). 


What are the pairs used to represent the nonnegative integers less than 12 when they 
are represented by the ordered pair where the first component is the remainder of the 
integer upon division by 3 and the second component is the remainder of the integer 
upon division by 4? 

Solution: We have the following representations, obtained by finding the remainder of 
each integer when it is divided by 3 and by 4: 


0 = 

{0,0) 

4 = 

(1,0) 

8 = 

(2,0) 

1 = 

an 

5 = 

(2.1) 

9 = 

(0.1) 

2 = 

(2,2) 

6 = 

(0, 2) 

10 = 

(1,2) 

3 = 

(0,3) 

7 = 

(1,3) 

11 - 

(2,3), m 

To perform arithmetic 

with 

large 

integers, we 

select moduli mj.mj. m„. 


where each m, is an integer greater than 2, gcd{m„ ntj) = 1 whenever i ^ /, and 
m = iriimj m„ is greater than the result of the arithmetic operations we want to 
carry' out. 

Once we have selected our moduli, we carry out arithmetic operations with large 
integers by performing componentwise operations on the n-tuples representing these 
integers using their remainders upon division by m;, i - 1,2,..., n Once we have 
computed the value of each component in the result, we recover its value by solving a 
system of n congruences modulo m„i - 1,2,..,, n, This method of performing arith¬ 
metic with large integers has several valuable features. First, it can be used to perform 
arithmetic with integers larger than can ordinarily be carried out on a computer. Second, 
computations with respect to the different moduli can be done in parallel, speeding up 
the arithmetic. 


Suppose that performing arithmetic with integers less than 100 on a certain processor is 
much quicker than doing arithmetic with larger integers. We can restrict almost all our 
computations to integers less than 100 if we represent integers using their remainders 
modulo pairwise relatively prime integers less than 100, For example, we can use the 
moduli of 09, 98, 97, and 95. (These integers are relatively prime pairwise, since no 
two have a common factor greater than 1.) 
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By the Chinese Remainder Theorem, every' nonnegative integer less than 99 -98 
97 * 95 = 89.403.930 can be represented uniquely by its remainders when divided by 
these four moduli. For example, we represent 123,684 as (33,8, 9, 89), since 123,684 
mod 99 = 33, 123,684 mod 98 = 8, 123,684 mod 97 = 9, and 123,684 mod 95 = 
89. Similarly, we represent 413,456 as (32,92, 42, 16). 

To find the sum of 123,684 and 413,456, we work with these 4-tuples instead of 
these two integers directly. We add the 4-tuples componentwise and reduce each coni' 
ponent with respect to the appropriate modulus. This yields 

(33.8,9,89) + (32,92,42,16) 

- (65 mod 99, 100 mod 98,51 mod 97,105 mod 95) 

- (65,2,51, 10). 

To find the sum, that is, the integer represented by (65, 2, 51, 10), we need to solve 
the system of congruences 

x -- 65 (mod 99) 
x = 2 (mod 98) 
x = 51 (mod 97) 
x = 10 (mod 95) 

It can be shown (see Exercise 29) that 537,140 is the unique nonnegative solution 
of this system less than 89,403,930. Consequently, 537,140 is the sum. Note that it is 
only when we have to recover the integer represented by (65, 2, 51, 10) that we have 
to do arithmetic with integers larger than ! 00. ■ 


Particularly good choices for moduli for arithmetic with large integers are sets of 
integers of the form 2*’ - 1, where k is a positive integer, since it is easy to do binary 
arithmetic modulo such integers, and since it is easy to find sets of such integers that 
are pairwise relatively prime, [The second reason is a consequence of the fact that 
ged{2“ -1. 1) = 2* ' J '" h ' - 1, as Exercise 31 shows. ] Suppose, for instance, that we 

can do arithmetic with integers less than 2 55 easily on our computer, but that working 
with larger integers requires special procedures. We can use pairwise relatively prime 
moduli less than 2 35 to perform arithmetic with integers as large as their product. For 
example, as Exercise 32 show's, the integers 2 33 - 1, 2 34 - 1, 2 33 - 1, 2 M - 1, 2 29 - i. 
and 2 2j - 1 are pairwise relatively prime. Since the product of these six moduli exceeds 
2 lg4 , w f e can perform arithmetic with integers as large as 2 iS4 (as long as the results do 
not exceed this number) by doing ari thmetic modulo for each of these six moduli, none 
of which exceeds 2 35 , 

PSEUDO PRIMES 


In Section 2.3 we showed that an integer n is prime when it is not divisible by any 
prime p with p <r Jn. Unfortunately, using this criterion to show that a given integer 
is prime is inefficient. It requires that we find all primes not exceeding jn and that we 
carry out trial division by each such prime to see whether it divides n. 

Are there more efficient ways to determine whether an integer is prime? Ancient 
Chinese mathematicians believed that n was prime if and only if 

2 fl “ 1 = ] (mod n). 
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If this were true, it would provide an efficient primalrty test. Why did they believe 
this congruence could be used to determine whether an integer is prime? First, they 
observed that the congruence holds whenever n is prime. For example, 5 is prime 
and 

2 5_l - 2 4 = 16 = 1 (mod 5). 

Second, they never found a composite integer n for which the congruence holds. The 
ancient Chinese were only partially correct. They were correct in thinking that tbe com 
gruence holds whenever n is prime, but they were incorrect in concluding that n is 
necessarily prime if the congruence holds. 

The great French mathematician Fermat showed that the congruence holds when 
n is prime. He proved the following, more general result. 

THEOREM 5 FERMAT’S LITTLE THEOREM If p is prime and a is an integer not divisible by 

p, that 

a p ~' 1 = 1 (mod p). 

Furthermore, for every integer a we have 
a p = a (mod p). 


The proof of Theorem 5 is outlined in Exercise 17 at the end of this section. 

Unfortunately, there are composite integers n such that 2 n ~' = 1 (mod n). Such 
integers are called pseudoprimes. 


EXAMPLE 9 The integer 341 is a pseudoprime since it is composite (341 = 11 • 31) and as Exercise 

23 shows 

2 340 - 1 (mod 341), ■ 


Although the ancient Chinese were wrong, pseudoprimes are relatively rare. Their 
scarcity—and the even greater scarcity of integers that pass more delicate tests that 
begin by determining whether an integer is a pseudoprime—can be used as the basis 
for efficient probabilistic primality tests. Such tests can be used to quickly show that 
it almost certainly is the case that a given integer is prime. (More precisely, these tests 


Pierre de Fermat (1601-1665). Pierre de Fermat, one o f the most important mathematicians of the seven- 
teenfh century. was a lawyer by profession. He is the most famous amateur mathematician in history. Fermat 
published little ot Ins mathematical discoveries. It is through his correspondence with other mathematicians 
that we know of his work. Fermat was one of the inventors of analytic geometry and developed some of 
the fundamental ideas of calculus Fermat, along with Pascal, gave probability theory a mathematical basis. 
Fermat formulated what is now the most famous unsolved problem in mathematics, Fie asserted that the 
equation _r" t y* = i" has no nontrivial positive integer solutions when n is an integer greater than 2- For 
more than .100 years, no proof (or counterexample) was found. It) his copy of the works of the ancient Greek 
mathematician Diuphamus, Fermat wrote that he had a proof but that it would not fit in the margin. Because 
the first proof, to unit by Andrew Wiles in 1994, relief ©n sophisticated, modern mathematics, most people 
think that Fermat thought lie had a proof, hut it was incorrect. However, he mav hnvc beers tempting Cithers 
to look lor a proof, nut being able to find one himself. 
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show that the probability that an integer that passes a series of tests is prime is close to 
1; see Chapter 4 for a discussion of probability.) These probabilistic primarily tests can 
be used, and are used, to find large primes extremely rapidly on computers. 


1’1'BHC KEY CRYPTOGRAPHY 


In Section 2.3 we introduced methods for encrypting messages based on congruences, 
M W When these encryption methods are used, messages, which are strings of characters, are 
translated into numbers. Then the number for each character is transformed into another 
number, either using a shift or an affine transformation modulo 26. These methods are 
examples of private key cryptosystems. Knowing the encryption key lets you quickly 
find the decryption key. For example, when a shift cipher is used with encryption key 
k, a number p representing a letter is sent to 

c - (p + k) mod 26. 

Decryption is carried out by shifting by that is, 
p = (c - k) mod 26. 

When a private key cryptosystem is used, a pair of people who wish to communicate 
in secret must have a separate key. Since anyone knowing this key can both encrypt and 
decrypt messages easily, these two people need to securely exchange the key. 

in the mid-1970s, cryptolngists introduced the concept of public key cryptosys¬ 
tems. When such cr, ptosystems are used, knowing how to send someone a message 
does not help you decrypt messages sent to this person. In such a system, every person 
can have a publicly known encryption key. Only the decryption keys are kept secret, 
and only the intended recipient of a message can decrypt it, since the encryption key 
does not let someone find the decry ption key without an extraordinary amount of work 
(such as 2 billion years of computer time). 

In 1976, three researchers at MI.T.— Ron Rivest, Adi Shamir, and Lett 
Adleman—introduced a public key cryptosystem, known as the RSA system, from 
the initials of its inventors. The RSA cryptosystem is based on modular exponentiation 
modulo of the product of two large primes. Each individual has an encryption key 
consisting of a modulus n = pq, where p and q are large primes, say, with 200 digits 
each, and an exponent e that is relatively prime to (p - Y)(q - 1). To produce a usable 
key, two large primes must be found. This can be done quickly on a computer using 
probabilistic primarily tests, referred to earlier in this section. However, the product of 
these primes n = pq, with approximately 400 digits, cannot be factored in a reasonable 
length of time. As we will see, this is an important reason why decryption cannot be 
done quickly without a separate decryption key. 


RSA ENC RYPTION 


In the RSA encryption method, messages are translated into sequences of integers. This 
can be done by translating each letter into an integer, as is done with the Caesar cipher. 
These integers are grouped together to form larger integers, each representing a block of 
letters. The encryption proceeds by transforming the integer M, representing the plain¬ 
text (the original message), to an integer C representing the ciphertext (the encrypted 
message), using the function 

C = M r mod n. 
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EXAMPLE 10 


EXAMPLE 11 


(To perform the encryption, we use an algorithm for fast modular exponentiation, such 
as that described in Exercise 14 in the Supplementary Exercises at the end of this chap¬ 
ter.) We leave the encrypted message as blocks of numbers and send these to the in¬ 
tended recipient. 

Example 10 illustrates how RSA encryption is performed. For practical reasons 
we use small primes p and q in this example, rather than primes with 100 or more 
digits. Although the cipher described in this example is not secure, it does illustrate the 
techniques used in the RSA cipher. 


Encrypt the message STOP using the RSA cryptosystem with p =■ 43 and q = 59, so 
that n = 43 ■ 59 = 2537, and with e = 13. Note that 

gcd(e, (p - 1 )(q - 1)) = ged(13, 42 ■ 58} = 1. 

Solution: We translate the letters in STOP into their numerical equivalents and then 
group the numbers into blocks of four. We obtain 

1819 1415. 

We encrypt each block using the mapping 
C — M 13 mod 2537. 

Computations using fast modular multiplication show that 1819 13 mod 2537 ~ 2081 
and 1415 13 mod 2537 = 2182, The encrypted message is 2081 2182, ■ 

RSA DECRYPTION 

The plaintext message can be quickly recovered when the decryption key d, an inverse 
of e modulo ( p - 1 ){q -■ 1), is known. [Such an inverse exists since gcd(e. Ip - 1) 
(<?-!)) = I.] To see this, note that if de = 1 (mod (p - 1 ){q - 1)). there is an integer 
k such that do = 1 + k(p - 1 )(q - 1), ft follows that 

C d = tv/ f = M de - 

By Fermat’s Little Theorem [assuming that gcd(M, p) ~ gcd(M, q) - 1, which holds 
except in rare cases], it follows that M p ~ 1 = 1 (mod p) and M v ~ ] = 1 (mod q). Con¬ 
sequently, 

C d = M • (A/' T' y =M- 1 - M (mod p) 
and 

C d = M ■ l) = Af ■ 1 = M (mod q). 

Since gcd(p, q) = l, it follows by the Chinese Remainder Theorem that 
C d : M (mod pq). 

Example 11 illustrates how to decrypt messages sent using the RSA cryptosystem. 


We receive the encrypted message 0981 0461. What is the decrypted message if it was 
encrypted using the RSA cipher from Example 10? 
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Solution: The message was encrypted using the RSA Cryptosystem with n = 43 ■ 59 
and exponent 13. As Exercise 4 shows, d - 937 is an inverse of 13 modulo 42 ■ 58 = 
2436. We use 937 as our decryption exponent, Consequently, to decrypt a block C, we 
compute 

P - C 9}1 mod 2537, 

To decrypt the message, we use the fast modular exponentiation algorithm to compute 
0981 9 ’ 1 mod 253? = 0704 and 0461 93 mod 2537 = 1115. Consequently, the nu¬ 
merical version of the original message is 0704 1115. Translating this back to English 
letters, we see that the message is HELP. ■ 


RSA AS A PUBLIC KEY SYSTEM 


Why is the RSA cryptosystem suitable for public key cryptography? When we know 
W€b the factorization of the modulus n, that is, when we know p and q, we can use the 
Euclidean algorithm to quickly find an exponent d inverse to e modulo (p - 1 ){q - 1}. 
This lets us decrypt messages sent using our key. However, no method is known to 
decrypt messages that is not based on finding a factorization of n, or that does not also 
lead to the factorization of r. The most efficient factorization methods known (as of 
1999) require billions of years to factor 400-digit integers. Consequently, when p and 
q are 200-digit primes, messages encrypted using n = pq as the modulus cannot he 
found in a reasonable time unless the primes p and q are known. 

Active research is under way to find new ways to efficiently factor integers. Integers 
that were thought, as recently as several years ago, to be far too large to be factored in 
a reasonable amount of time can now be factored routinely. Integers with more than 
100 digits, as well as some with more than 150 digits, have been factored using team 
efforts. When new factorization techniques are found, it will be necessary to use larger 
primes to ensure secrecy of messages. Unfortunately, messages that were considered 
secure earl ier can be saved and subsequently decrypted by unintended recipients w'hen 
it becomes feasible to factor the n - pq in the key used for RSA encryption. 

The RSA method has been implemented and is used for some particularly sensitive 
applications. However, the most commonly used cryptosystem is a private key system 
known as DES (an acronym for the Data Encryption Standard). When DES is used, en¬ 
cryption and decry ption can be performed extremely rapidly on a computer. Although 
some people believe that messages encrypted using DES can be broken by experts, DES 
is considered sufficiently secure in most situations. The use of public key cryptography, 
via the RSA system, is growing, but when the RSA system is used, encryption and 
decryption are too slow (using the current generation of computers) for many applica¬ 
tions. However, there are applications that use both private key and public key systems. 
For example, a public key cryptosystem, such as RSA, can be used to distribute private 
keys to pairs of individuals when they wish to communicate. These people then use a 
private key system, such as DES, for encryption and decryption of messages. 


Exercises 


f. Express Jfrc greatest commaii di visor of each of tfie fb 1 - 

lowing pairs nf integers as a linear combination of these 
integers. 


a) Hi, It 
d) 34,55 
g) 123,2347 


b) 21,44 
e) 117,213 
hi 3454.4666 


cl 36,48 
f) 0, 223 
i) 9999, 11111 
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2. Express the greatest common divisor of each of the fol¬ 
lowing pairs of integers as a linear combination of these 
integers. 

a) 9. 11 b) 33.44 

c) 35.78 d) 21,55 

e) 101.203 f) 124,323 

g) 2002. 2339 h) 3457, 4669 

i) 10001.13422 

3. Show that 15 is an inverse of 7 modulo 26. 

4. Show that 937 is an inverse of 13 modulo 2436. 

5. Find an inverse of 4 modulo 9. 

6. Find an inverse of 2 modulo 17. 

7. Find an inverse of 19 modulo 141. 

Find an inverse of 144 modulo 233, 

*9, Show that if a and m are relatively prime positive inte¬ 
gers, then the inverse of a modulo m is unique modulo 
m, | Hint: Assume that there are two solutions b and c 
of the congruence ax = 1 (mod m). Use Theorem 2 to 
show that b = c (mod m ).] 

10. Show that an inverse of a modulo m docs not exist if 
gedffl. m) > 1. 

11. Solve the congruence 4 j - 5 (mod 9). 

12. Solve the congruence 2.v s 7 (mod 17). 

*13. Show that if m is a positive integer greater 
than 1 and ac 3 be (mod m), then a ^ b 
mod m/gcd(c, m). 

14. a) Show' that the positive integers Jess than H, except 

1 and 10, can be split into pairs of integers such 
that each pair consists of integers that are inverses 
of each other modulo 11. 
b) Use pair (a) to show that 10! = --1 (mod 11). 

15, Show that if p is prime, the only solutions of jr = 

! (mod p) tire integers v such that x = 1 (mod p} and 
x “ - I (mod p), 

*16. a) Generalize the result in part (a) of Exercise 14; that 
is, show that ifp is a prime, the positive integers less 
than p, except 1 and p - I, can be split into (p - 
3W2 pairs of integers such that each pair consists of 
integers that are inverses of each other. (Him; Use 
the result of Exercise 15.) 

b) From part (a) conclude that (p - I)! s -1 
(mod p) whenever p is prime. This result is known 
as Wilson’s theorem. 

c) What can we conclude if n is a positive integer such 
that in 1)1 T -1 (mod nU 

*17. This exercise outlines a proof of Fermat's Little Theo¬ 
rem. 

a I Suppose that a is not divisible by the prime 
p, Show that no two of the integers I ■ a, 

2 ■ (i— ,(p IV/ are congruent modulo p, 
b) Conclude from part (a) that the product of 

1,2- p - 1 is congruent modulo p to the product 

of«. 2«. [p - 1 )o. Use this to show that 

(/’ 14-- m" : (/> ■ 14 (mod p). 


e) Use Wilson’s theorem (proved in Exercise 16) to 
show that a p 1 = i (mod p) if p i a. 
d) Use part (c) to show that a p s (mod p i for ail 
integers a. 

18. Use the Chinese Remainder Theorem to show that an 
integer a, with 0 < a < m - mi My -■ ■ m n , where the 

integers np.mi _ m„ are pairwise relatively prime. 

can be represented uniquely by the n-tuple (a mod m ; , 
a mod nr__ a mod m„). 

*19. Let wi|. j« 2 , ..., m„ be pairwise relatively prime in¬ 
tegers greater than or equal to 2. Show that if a = 

b (mod t»f) for i = 1,2. n, then a - f>(mod mi, 

where m = immi ■ ■■ m„, 

*20. Complete the proof of the Chinese Remainder Theorem 
by showing that the simultaneous solution of a system 
of linear congruences modulo pairwise relatively prime 
integers is unique modulo the product of these moduli. 
(Hint; Assume that x and y are two simultaneous solu¬ 
tions, Show that m, | x - y for all i. Using Exercise 19, 
conclude that rn = mpn, ■ m„ \ x - v.) 

21. Which integers leave a remainder of 1 when divided by 
2 and also leave a remainder of 1 when divided by 3? 

22. Which integers are divisible by 5 but leave a remainder 
of I when divided by 3? 

23. a) Show that 2 M0 = 1 (mod 11) by Fermat’s Little 

Theorem and noting that 2 m - (2 IU )- W . 

b) Show that 2 m> = 1 (mod 31) using the fact that 

2-Mu = j-yijt# _ 

c) Conclude front parts (a) and (b) that 2- 14 ’ 1 = 

1 (mod 341), 

24. a) Use Fermat’s Little Theorem to compute 3 W ’ mud 

5.3 : ’ o: mod 7, and 3 TO mod II. 
b) Use your results from part (a) and the Chinese Re¬ 
mainder Theorem to find 3® mod 385. (Note that 
385 = 5 ■ 7 ■ 11.) 

25. a) Use Fermat's Little Theorem to compute 5 im - 1 mod 

7. 5 :o °-' mod 11, and 5® 0 - 1 mod 13. 
b) Use your results from pan (a) and the Chinese Re¬ 
mainder Theorem to find 5™ ;1 mod IDOL (Note 
that 1001 =711- 13.) 

26. Find the nonnegative integer a less than 28 represented 
by each of the following pairs, where each pair repre¬ 
sents (a mod 4, it mod 7) 

a) (0,0) b) (LA) c) (1, 1) 

d) (2, 1) el (2. 2) f) (0. 3) 

g> (2. 0) h) (3. 5) i) (3.6) 

27. Express each nonnegative integers less than 15 using 
the pair [a mod 3, a mod 5). 

28. Explain how to use the pairs found in Exercise 27 to 
add 4 and 7. 

29. Solve the system of congruences that arises in Exam¬ 
ple 8. 

*30. Show that tfn and b are positive integers, then (2"' - 
11 mod!-' 1 ' - 1) - - 1. 
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**31. Use Exercise 30 to show that if a and b are posi¬ 
tive integers, then gcd(2" -1,2* - 1) = 2 scd!aW - 1. 
[Him: Show that the remainders obtained when the Eu¬ 
clidean algorithm is used to compute gcd(2“ — 1,2*' — 
I) are of the form 2" - 1, where r is a remainder 
arising when the Euclidean algorithm is used to find 
gcd(a, h).] 

32. Use Exercise 31 to show ihalthe integers 2 is - 1, 2 14 - 
I, 2 51 - l, 2 }l - 1,2® - 1, and 2’ 1 - 1 are pairwise 
relatively prime. 

33. Show that if p is an odd prime, then every divisor of 
the Morse tine number 2 P - 1 is of the form 2 kp + 1, 
where k is a non negative integer. ( Hint: Use Fermat’s 
Little Theorem and Exercise 3!.) 

34. Use Exercise 33 to determine whether - 2 15 - 
1 - 8191 and Af B - 2 s - 1 - 8,388,607 are prime. 

*35. Show that we can easily factor n when we know that n 
is the product of two prunes, p and q, and we know the 
value of (p - 1 )(g - 1), 

36, Encrypt the message ATTACK using the RSA system 
with n = 43 ■ 59 and e — 13, translating each letter 
into integers and grouping together pairs of integers, as 
done in Example 10, 

37, What is the original message encrypted using the RSA 
system with n = 43 -59 and e = 13 if the encrypted 
message is 0667 1947 0671? (Note: Some computa¬ 
tional aid is needed to do this in a realistic amount of 
time.) 

If m is a positive integer, the integer a is a quadratic 
residue of m if gedtd, m) = ! and the congruence ,t 2 = 
a (mod m) has a solution. In other words, aquadratie residue 
of m is an integer relatively prime to m which is a perfect 
square modulo m. For example. 2 is a quadratic residue of 7 
since ged(2, 1) = 1 and 3^2 (mod 7) and 3 is a quadratic 
nonresidue of 7 since ged(3, 7) = I and x l = 3 (mod 7) has 
no solution. 

38, Which integers are quadratic residues of 117 


39. Show that if p is an odd prime and a is an integer not 
divisible by p, then the congruence x 2 = a (mod p) has 
either no solutions or exactly two incongruent solutions 
modulo p. 

40. Show that if p is an odd prime, then there are exactly 
(p - l)/2 quadratic residues of p among the integers 

1,2 _ p- 1 . 

Up is an odd prime and a is an integer not divisible by p, the 

Legendre symbol (—is defined to be I if a is a quadratic 

\P / 

residue of p and -1 otherwise. 

41. Show that if/> is an odd prime and a and b are integers 
with a = fj(niodp), then 



42. Prove that if p is an odd prime and a is a positive integer 
not divisible by p, then 

^ j = a L/, “ ly3 (modp). 

43. Use Exercise 42 to show' that if p is an odd prime and 
a and h are integers not divisible by p. then 



44, Show that if p is an odd prime, then - I is a quadratic 
residue of p if p = 1 (mod 4) and 1 is not a quadratic 
residue atp if p = 3 (mod4), {Him.- Use Exercise 42.) 

45, Find all solutions of the congruence ,r 2 = 29 (mod 35). 
(Him: find the solutions of this congruence modulo 5 
and modulo 7, and then use the Chinese Remainder 
Theorem.) 

46, Find all solutions of the congruence x 2 = 16 (mod 105). 
{Hint: Find the solutions of this congruence modulo 3, 
modulo 5, and modulo 7, and then use the Chinese 
Remainder Theorem. ) 


2.6 

Matrices 


INTRODUCTION 


Matrices are used throughout discrete mathematics to express relationships between 
elements in sets. In subsequent chapters we will use matrices in a wide variety 
of models. For instance, matrices will be used in models of communications net¬ 
works and transportation systems. Many algorithms will be developed that use these 
matrix models. This section reviews matrix arithmetic that will be used in these 
algorithms. 
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DiKMUOW t A matrix is a rectangular array of A with 

rows and n ooiumns is called tn«xa matrix. Tt«r p&tal.flf matrix itmmtmm 


matrix with the same number of rows as cotamus is called i^Harv. Ttoo nuancea an 
equal if they have tbe same number of rows aid the same number of etdiann* and 
the corresponding entries in every position are equal. 


EXAMPLE 1 


The matrix 


1 1 
0 2 
1 3 


is a 3 X 2 matrix. 


We now introduce some terminology about matrices. Boldface uppercase letters 
will be used to represent matrices. 


DEFINITION Z. Let 

CI|1 fin "• «!„ 

«21 «22 


.toll &tt2 Man 


The ith raw of A is the 1 Xu matrix [«,i, oq, ■■■, <H*]- The yth cohaim of A is the 
n X 1 matrix 


V2j 


a* j 


The (t. y)th element or entry of A is the element a^, that is, die number in the rth 
row and ;th column of A. A convenient shorthand notation for expressing the ma¬ 
trix A is to write A = [< 2 ;y], which indicates that A is die matrix with its (i, jftft 
element equal to a,;. 


MATRIX ARITHMETIC 


The basic operations of matrix arithmetic will now be discussed, beginning with a def¬ 
inition of matrix addition. 


DEFINITION 3. Let A = [a,j] and B = [by] be. m X n matrices. The of A 
and B, denoted by A + B, is the m X n matrix that has m ; + b,j as its (i, fltb element. 
In other words, A + B = [a t! + b, ; ]. 
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The sum of two matrices of the same size is obtained by adding elements in the corre¬ 
sponding positions. Matrices of different sizes cannot be added, since the sum of two 
matrices is defined only when both matrices have the same number of rows and the 
same number of columns. 



ri 

0 

-r 

f 3 4 -r 


r 4 

4 -2] 

EXAMPLE 2 

We have 1 2 

2 

-3 

+ 1-3 0 

= 

3 

-1 -3. 


h 

4 

0 

[-1 1 2 


2 

5 2 j 


We now discuss matrix products. A product of two matrices is defined only when 
the number of columns in the first matrix equals the number of rows of the second 
matrix. 


DEFINITION 4, Let A be an m x k matrix and B be a t X nmatrix. The prvduci 
of A and B, denoted byAB, is ti* m x n mwtrf* aqua] tethe sum of 

tbe ptodiaets of ttie C0f^e$|>ei8itding edisxieitsts ftom :T A and the jfeeohann 

of B. In other words, if AB = [e/j], (ben 

k 

Cij = a.ibij + Oizhj + ■ • • + aubjrj * dttbkj. . 

("l 


(n Figure 1 the colored row of A and the colored column of Ft are used to compute 
the element qj of AB. The product of two matrices is not defined when the number of 
columns in the first matrix and the number of row's in the second matrix are not the same. 
We now give some examples of matrix products. 


EXAMPLE 3 Let 

n ° 4 1 [2 4' 

19 1 1 4 * i 

A = I and B - 1 1 I 

[0 2 2j l 3 

Find AB if it is defined. 


«n 

flzt 

a n ■ 

£i?2 

■ Tfc 
' tbi 

_ 

bn • 

■■ b>, 

" b\n 


+1) 

C M 

■■ C,/ 





bn 

b-7 2 ■ 

*■ hi 

■■ h„ 


£’21 

Q: 

Cy„ 

«n 

an 

' a>k 


bn 

bid ■ 

■ b kJ 

' ’ bkn 



c 

j ■ 

&tn\ 

a m 2 ■ 

■ a mk 





/ml 

C m 2 ' 

Cmn _ 


FIGURE 1 The Product of .4 = |Vi (; ] and B - 
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EXAMPLE 4 


Solution: Since A is a 4 x 3 matrix and B is a 3 x 2 matrix, the product AB is de¬ 
fined and is a 4 X 2 matrix. To find the elements of AB, the corresponding elements 
of the rows of A and the columns of B are first multiplied and then these products 
are added. For instance, the element in the (3, l)th position of AB is the sum of the 
products of the corresponding elements of the third row of A and the first column of 
B; namely, 3-2+11+0-3 = 7. When all the elements of AB arc computed, we 
see that 

14 4 


8 2j 


Matrix multiplication is not commutative. That is, if A and B are two matrices, it 
is not necessarily true that AB and BA are the same. In fact, it may be that only one 
of these two products is defined. For instance, if A is 2 x 3 and B is 3 X 4, then AB is 
defined and is 2 X 4; however, BA is not defined, since it is impossible to multiply a 
3 X 4 matrix and a 2 X 3 matrix. 

In general, suppose that A is an m X n matrix and B is an r X s matrix. Then 
AB is defined only when n = r and BA is defined only when s - m. Moreover, 
even when AB and BA are both defined, they will not be the same size unless 
m - n = r - s. Hence, if both AB and BA are defined and are the same size, then 
both A and B must be square and of the same size, Furthermore, even with A and B 
both n X n matrices, AB and BA are not necessarily equal, as the following example 
demonstrates. 


Let 


A = 


1 1 

2 I 


and 


Does AB - BA' 1 
Solution: We find that 


3 2) 
5 3 


AB = 

Hence, AB + BA, 


and 


B = 


2 I 

1 1 


BA = 


4 3 
3 2 


ALGORITHMS FOR MATRIX MULTIPLICATION 

The definition of the product of two matrices leads to an algorithm that computes 
the product of two matrices. Suppose that C = [c, ; ] is the m X n matrix that is the 
product of the m x k matrix A = [«, ; ] and the k X n matrix B = [h/,]. The algo¬ 
rithm based on the definition of the matrix product is expressed in pseudocode in 
Algorithm 1, 
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EXAMPLE 5 


EXAMPLE t 


ALGORITHM 1 Matrix Multiplication, 

procedure matrix multiplication(A, B: matrices) 

for i : = 1 to m 
begin 

for j 1 to n 
begin 

Cfj := 0 

for q 1 to k 

c ‘j '■' c ‘j f 

end 

end {C = Idj] is the product of A and B} 


We can determine the complexity of this algorithm in terms of the number of additions 
and multiplications used. 


How many additions of integers and multiplications of integers are used by Algorithm 
I to multiply two n x n matrices with integer entries'? 

Solution: There are rr entries in the product of A and B. To find each entry 1 requires a 
total of n multiplications and n - 1 additions. Hence, a total of n 3 multiplications and 
n 2 (n - I) additions are used. ■ 


Surprisingly, there are more efficient algorithms for matrix multiplication than that 
given in Algorithm 1. As Example 5 shows, multiplying two n x n matrices directly 
from the definition requires O(jr') multiplications and additions. Using other algo¬ 
rithms, two n >: n matrices can be multiplied using Oin ) multiplications and addi¬ 
tions. (Details of such algorithms can be found in the references given in the suggested 
readings at the end of the book.) 

There is another important problem involving the complexity of the multiplication 
of matrices. How should the product A] A; - A,, be computed using the fewest mul¬ 
tiplications of integers, where A t ,A 2 .A, are m, X m 2 , m 2 X m 3 . m„ x m„ +l 

matrices, respectively, and each has integers as entries? (Since matrix multiplication is 
associative, as shown in Exercise 13 at the end of this section, the order of the multipli¬ 
cation used does not matter.) Before studying this problem, note that mj multipli¬ 
cations of integers are perforated to multiply an m\ x m 2 matrix and an m 2 x m : . matrix 
using Algorithm 1 (see Exercise 23 at the end of this section). The following example 
illustrates this complexity problem. 


In which order should the matrices A|, A 2 , and A 3 —where A] is 30 X 20, A 2 is 20 X 40, 
and A : i is 40 x 10, all with integer entries—be multiplied to use the least number of 
multiplications of integers? 

Solution: There are two possible ways to compute A S A]A 5 These are A : (A 2 A :1 ) and 
(A : Ai)A :! . 
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Jf A; and A 3 are first multiplied, a total of 20 ■ 40 * 10 - S0f_>0 multiplications of 
integers are used to obtain the 20 X 10 matrix AjAi, Then, to multiply A| and A 3 A 3 
requires 30 ■ 20 ■ 10 = 6000 multiplications. Hence, a total of 

8000 + 6000 = 14,000 

multiplications are used. On the other hand, if A| and A; are first multiplied, then 
30- 20-40 = 24,000 multiplications are used to obtain the 30 X 40 matrix Aj A 3 , Then, 
to multiply A(A 2 and A, requires 30 X 40 X !0 = 12,000 multiplication. Hence, a 
total of 

24,000 + 12,000 = 36,000 
multiplications are used. 

Clearly, the first method is more efficient. ■ 


Algorithms for determining the most efficient way to multiply n matrices are discussed 
in the suggested readings listed at the end of the book. 

TRANSPOSES AND POWERS OF MATRICES 


We now introduce an important matrix with entries that are zeros and ones. 

DEFINITION 5. The identity matrix of order n is the n X n matrix I n = [o i; ], 
where = 1 if f = j and 5y = 0 if i ¥= j. Hence 

1 0 O' 

0 1 - 0 


0 0 1 

Multiplying a matrix by an appropriately sized identity matrix does not change this 
matrix. In other words, when A is an m X n matrix, we have 

Al„ - I ffl A - A 

Powers of square matrices can be defined. When A ts an n x n matrix, we have 
A n = I„, A r -- AAA ■ ■ ■ A. 



The operation of interchanging the rows and columns of a square matrix is used in 
many algorithms. 

DEFINITION 6. Let A - [a,y] be an m x n matrix. Hie transpose of A, denoted 
by A*, is the n x m matrix obtained by interchanging the rows and columns of A, In 
other words, if A' = \b u }, then b u = a „ for i ~ 1,2,. n and j = 1,2,.... m. 


PDF created with pdfFactory trial version www.pdffactorv.com 






156 


2 J The Fundamentals; Algorithms, the Integers, and Matrites 


EXAMPLE 7 


EXAMPLE 8 



The transpose of the matrix 


1 2 

3 ] 

, i.s the matrix 

'l 

2 

4 n 

5 

4 5 

6J 

.3 

6 . 


Matrices that do not change when their rows and columns are interchanged are often 
important. 


BEHNITiON 7. A square matrix A is called symmetric iS A = A'. Thus A « 
[a, j) is symmetric if * «jt for all i and j with! s= i STtandl ^ js n. 


Note that a matrix is symmetric if and only if it is square and it is symmetric with 
respect to its main diagonal (which consists of entries that are in the ith tow and ith 
column for some t). This symmetry is displayed in Figure 2. 


fl ! ol 

The matrix ) 0 t is symmetric. ■ 

[o 1 oj 

ZERO-ONE MATRICES 


A matrix with entries that are either 0 or I is called a zero—one matrix. Zero-one 
matrices are often used to represent discrete structures, as we will see in Chapters 6 
and 7, Algorithms using these structures are based on Boolean arithmetic with zero-one 
matrices. This arithmetic is based on the Boolean operations V and A, which operate 
on pairs of bits, defined by 


hi A h 



if hi = £>2 = 1 
otherwise, 


b i V A 



if b[ = 1 or A = 1 
otherwise. 


DEFINITION 8, Let A - [<ay] and 8- [fey] he mx Rzm-tmeiMt«£*s.Then 
fas join of A and Bis Ate icro-as® malm enfey y fey. Hue Jehu of 

A l % A y' ft; Tfte meet of A iai Bfe te 2 ett>A»e mastix with 

ft, ,/)th end? % A fe.-j. The meet of A and Bis denoted by A A 8. 
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EXAMPLE 9 


EXAMPLE 10 


Find the join and meet of the zero-one matrices 


T 0 l 

B = [° 

0 

O 

o 

’ “ [l 

0. 


Solution: We find that the join of A and B is 


AyB 


l yO 

oyi 

ivo] _ [1 

*1 

Oyi 

ivi 

oyoj [l 

oj 


The meet of A and B is 


1 AO 

0 A 1 

1 A 0 

[0 0 0 

0 A 1 

1A 1 

0 AO 

[0 1 o 


We now define the Boolean product of two matrices. 



Note that the Boolean product of A and B is obtained in an analogous way to the or¬ 
dinary product of these matrices, but with addition replaced with the operation V and 
with multiplication replaced with the operation A. We give an example of the Boolean 
products of matrices. 


Find the Boolean product of A and B, where 


A - 


I 

0 

1 


0 

I 

0 


B = 


1 1 0 
0 1 1 


Solution: The Boolean product A O B is given by 


A OB 


(1 At)V(OAO) 
(OAl-)v(l AO) 
(lAl)y(OAO) 


{1 Al)y(0Al) 
(0 A1) y (1A I) 
(lAl)v(OAl) 


(lAO)y(OAlJ 
(OAO)yn Al) 
(1 AO)y(OAl) 


lyO lyO OyO 
oyo oyi Oyi 
lyO lyO OyO 

1 1 0 

Oil. 

1 1 0 
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Algorithm. 2 displays pseudocode for computing the Boolean product of two 
matrices. 


ALGORITHM 2 The Boolean Product. 

procedure Boolean product( A, B: zero-one matrices) 

for i : = 1 to m 
begin 

for j ; = 1 to n 

begin 

c J; :=0 

for q : = 1 to k 

Cij — C'ij V ( a iq ‘ h \bqj) 

end 

end {C = [t jj] is the Boolean product of A and B} 


We can also define the Boolean powers of a square zero-one matrix. These powers 
will be used in our subsequent studies of paths in graphs, which are used to model such 
things as communications paths in computer networks. 


DEFINITION 10. Let A be a square zero-one matrix and let r be a positive in* 
teger. The rth Boolean power of A is the Boolean product of r factors of A. The rife 
Boolean product of A is denoted by AM. Hence 

A« = A0AOA0---QA, . 


EXAMPLE 11 


(This is well defined since the Boolean product of matrices is associative.) We also 
define A ,0J to be L. 


Let A = 


0 0 ll 

1 0 0. Find AM for all positive integers n, 

110 


Solution: We find that 


A f2J = A 0 A - 


1 1 
0 0 
1 0 


0 

1 

1 


We also find that 



I 0 1 


1 I 1 

AM= A |2 ' O A = 

1 1 0 

1 1 1 

, A™ = A^0A = 

1 0 1 

1 1 1 
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Additional computation shows that 


A [i| = 


1 

1 

1 


1 1 
l 1 
1 1 


The reader can now see that A 1 " 1 = A fS ' for all positive integers n with n > 5. ■ 


The number of bit operations used to find the Boolean product of two n x n matrices 
can be easily determined. 


EXAMPLE 12 How many bit operations are used to find A 0 B, where A and B are n X n zero-one 

matrices? 

Solution: There are tv entries in A 0 B. Using Algorithm 2, a total of n O/fs and n ANDs 
are used to find an entry of A O B. Hence. 2 n bit operations are used to find each entry'. 
Therefore, 2n 3 bit operations are required to compute A 0 B using Algorithm 2. ■ 


Exercises 


1113 

1. Let A = 2 0 4 6 

.1 1 3 7. 

a) What size is A? 

b) What is the third column of A? 
e) What is the second row of A? 

d) What is the element of A in the (3,2)th position ? 

e) What is A' ? 

2. find A + B, where 
0 


a) A - 


b) A = 


L-4 

3. Find AB if 


41 

2|.B 

-3j 

5 6 

5 -2 


, B 


-1 3 

2 2 
2 -3 

-3 
0 


4 

oj 

9 -3 
-2 -1 


a) A = 


b) A - 


c) A 


4 -3 
3 -1 
0 -2 
-1 5 


, B 


3 -2 -1 
i 0 2 


-1 32-2 

0-14-3 


4. Find the product AB, where 


a) A = 


b) A - 


e) A 


0 1 


0 -1 -1 


B 


-I 1 
1 -3 


2 

1 1 

0 -r 
7 


0 

1 -1 
-1 0 

1 -1 2 

B = |—I 0 


i -r 


3 - 


L-3 -2 0 


2 j, B = 

-3J 

5. Find a matrix A such that 


-I 2 
0 3 


(Hint: Finding A requires that you solve systems of lin¬ 
ear equations.) 

6. Find a matrix A such that 


n 3 2 ' 


1 13 

{2 1 1 

A = 

1 0 3 

|4 0 3. 


-1 -3 7. 


7. Let A be an in X n matrix and let 0 be the m X n matrix 
that has all entries equal to zero. Show that A = 0 + 
A = A + I) 

8. Show that matrix addition is commutative; that is, show 
that if A and B are both m x n matrices, then A + B = 

B + A. 
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9. Show that matrix addition is associative; that is, show 
that if A, B, and C are ail m x n matrices, then A + 
(B + C) = (.4 + B) 4 C. 

10, Let A be a 3 A 4 matrix, B be a 4 X 5 matnx, and C 
be a 4 X 4 matrix. Determine which of the following 
products are defined and find the size of those that are 
defined. 

a) AB b) BA c) AC 

d> CA e) BC f) CB 

11. What do we know about the sizes of the matrices A and 
B if both of the products AB and BA are defined'? 

12, In this exercise we show that matrix multiplication is 
distributive over matrix addition. 

a) Suppose that A and B are m x k matrices and that C 
is a k X n matrix. Show that (A 4 B)C = AC + BC. 
b! Suppose that C is an m x k matrix and that A and S 
are k >: n matrices. Show that C( A+B) = CA 4CB, 

13. In this exercise we show that matrix multiplication is 
associative. Suppose that A is an m X p matrix, R is 
a p x k matrix, and C is a k X n matrix. Show that 
A(BC) = (AB)C, 

14. The nX it matrix A = f«j, | is called a diagonal matrix 
it a,j =0 when i ¥= j. Show that the product of two 
n x n diagonal matrices is again a diagonal matrix. Give 
a simple rule for detenu ini ng this product. 

15, Lei 


Find a formula for A", whenever n is a positive integer. 

16, Show that (A'/ = A. 

17, Let A and B he two n x a matrices. Show that 
a) (A 4 B)' = A' 4 B'. b) (AB)' = B'A' 

If A and B are n x n matrices with AB = BA = I,,, then B 
is called the inverse of A (this terminology is appropriate 
since such a matrix R is unique) and A is said to be invert¬ 
ible, The notation B = A 1 denotes that B is the inverse 

of A. 

18, Show that 

[2 3 -I 

12 1 

[-1 -1 3 

is the inverse of 

: 7 -8 51 

--4 5 -3 . 

!. I -1 !J 

19, Let A be a 2 X 2 matrix with 



Show that if ad - be 0, then 

[ d -b 

l = I ad - be ad - he 

i ~ t: 

[ ad - he ad - be j 

20, Let 



a) Find A (Hint: Use Exercise 19.) 

b) Find A\ 

c) Find (A -1 ) 3 . 

d) Use your answers to (b) and (c) to show that (A ’)• 
is the inverse of A 3 . 

21. Let A be an invertible matrix. Show that (A rt ) 1 = 
(A -1 )” whenever n is a positive integer. 

22. Let A be a matrix. Show that the matrix AA' is sym¬ 
metric. (Hint; Show that this matrix equals its trans¬ 
pose with the help of Exercise 17b.) 

23. Show that the conventional algorithm uses 
multiplications to compute the product of the mi x 
matrix A and the nr x m j matrix B. 

24. What is the most efficient Way to multiply the matrices 
A], At, and A 3 with sizes 

a) 20 X 50,50 x 10.10 x 40? 

b) 10 X 5,5 X 50,50 X 1? 

25. What is the most efficient way to multiply the matrices 
A ; , Aj, A;i, and A* if the dimensions of these matrices 
are 10 X 2. 2 x 5, 5 x 20, and 20 x 3, respectively? 

26. a) Show that the system of simultaneous linear equa¬ 

tions 

a ] [ X\ 4 J 12 X 2 4 ■ ■ ■ 4-fbnXfl — b\ 
ffjl -t] + rL;-L> + ■ ■ • +a.;„.r„ = b 7 


a n 1 jtj 4 + ■ ■ -+£i„ n jr 0 = b„ 

in the variables , xj.r„ can be expressed as 

AX = B, where A = [a,yj, X is an ft x 1 matrix 
with Xf the entry in its ith row, and B is an n x 1 
matrix with b, the entry in its ith row. 
b) Show that if the matrix A = [a,/] is invertible (as 
defined in the preamble to Exercise 18), then the 
solution of the system in part (a) can be found using 
the equation X = A _ 1 B, 

27, Use Exercises 18 and 26 to solve the system 

7xi - 8 x 2 4 5xj = 5 
-4x t 4 5x2 - 3 a 3 = -3 
V) - J; 4 r 3 .= 0 

28. Let 


A r. 


I’ ’ 

>0 1 


and 


[0 11 

11 0J- 
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Find 

a) A y B. b) A A B. 
c) A © B. 

2<L L.et 



1 0 1 

[o i r 

A = 

1 1 0 

and B = j i 0 1 


.0 0 1 . 

[l 0 1 . 


Find 

a) A y B, 

b) A A B. 
e) A © B. 

30. Find the Boolean product of A and B, where 


10 0 1 

0 10 1 

and B = 

1 ol 
0 1 

1 j 

1111 . 


1 0 


31. Let 


A 


'1 0 O' 

1 0 1 

0 1 0 


Find 

a) A [ A b) A P1 . CMVA^VA'- 11 . 


32. Let A be a zero-one matrix. Show that 
a .) A y A - A 

b) A A A = A. 

33. In this exercise we show that the meet and join opera¬ 
tions are commutative. Let A and B be m X n zero-one 
matrices. Show that 

a) A y B = By A. 

b) BAA = A A B. 

34. In this exercise we show that the meet and join opera¬ 
tions are associative. Let A, B, and C be m X n zero-one 
matrices. Show that 

a) (AyBiyC = A y (B y C). 

b) (A A B) A C = A A K A C), 

35. We will establish distributive laws of the meet over the 
join operation in this exercise. Let A, B, and C be m X n 
zero-one matrices. Show that 

a) A y (B A C) = (A y B) A (A y C). 

b) A A (B y C) - (A A B) y (A A C). 

36. Let A be an n x n zero-one matrix. Let 1 be 
the ft y n identity matrix. Show' that A 3 I - 
IO A = A. 

37. In this exercise we will show that, the Boolean product 
of zero-one matrices is associative. Assume that A is 
an m '< p zero-one matrix, B is a p '< k zero-one matri x. 
and C is a k X n zero-line matrix. Show that A 8 (B G 
C) = (A0 B) G C 


Key Terms and Results 

TERMS 

algorithm: a finite set of precise instructions for perform¬ 
ing a computation or solving a problem, 
searching algorithm: the problem of locating an element 
in a list 

linear search algorithm: a procedure for searching a list 
clement by element 

binary search algorithm: a procedure for searching an 
ordered list by successively splitting the list in half 
time complexity: the amount of time required for an al¬ 
gorithm to solve a problem 

space complexity: the amount of storage space required 
for an algorithm to solve a problem 
worst-case time complexity: the greatest amount of time 
required for an algorithm to solve a problem of a given 
size 

average-case time complexity: the average amount of 
time required for an algorithm to solve a problem of a 
given size 

a J h (a divides b): there is an integer c such that b - ac 
prime: a positive integer greater than 1 with exactly two 
positive integer divisors 


composite: a positive integer greater than ) that is not 
pnme 

Mersenne prime; a prime of the form 2’’ - 1, where p is 
prime 

gcd(«, b) (greatest common divisor of a and by. the 
largest integer that divides both a and b 
relatively prime integers: integers a and b such that 
gedfo, b) = 1 

pairwise relatively prime integers: a set of integers with 

the property that every pair of these integers is relatively 
prime 

lcm(n, b) (least common multiple of a and b); the small¬ 
est positive integer that is divisible by both a and b 
a mod b: the remainder when the integer a is divided by 
the positive integer b 

a = b (mod mi (a is congruent to b modulo m); a - b is 

divisible by m 

encryption: the process of making a message secret 
decryption: the process of returning a secret message to 
its original form 

n = (ajfOi -1 ■ ■ ■ the base b representation of n 

binary representation: the base 7 representation of an in¬ 
teger 
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hexadecimal representation: the base 16 representation 
of an integer 

linear combination of a and b with integer coeffi¬ 
cients; a number of the form m + ib where f and ! are 
integers 

inverse of a modulo m; an integer a such that Sa = 
1 (mod m) 

linear congruence: a congruence of the form ax - 
b (mod m) where x is a variable 
pseudoprime to the base 2 : a composite integer n such 
that 2” 1 = I (mod n) 

private key encryption: encryption where both encryp¬ 
tion keys and decryption keys must be kept secret 
public key encryption: encryption where encryption 
keys are public knowledge, but decryption keys are kept 
secret 

matrix: a rectangular array of ji umbers 
matrix addition: seepage 151 
matrix multiplication: see page 152 
I„ (identity matrix of order n i: the a x n matrix that has 
entries equal to 1 on its diagonal and Os elsewhere 
A' (transpose of A): the matrix obtained from A by inter¬ 
changing the raws and columns 
symmetric: a matrix is symmetric if it equal s its transpose 
zero-one matrix: a matrix with each entry equd to either 
0 or 1 

A \/ B (the join of A and B): see page 156 
A A B (the meet of A and B): see page 156 
A 0 B (the Boolean product of A and B): see page 157 

RESULTS 

The linear and binary search algorithms; (given in Sec¬ 
tion 2 . 1 ). 


The Fundamental Theorem of Arithmetic; Every pos¬ 
itive integer can be written uniquely as the product of 
primes, where the prime factors are written in order of 
increasing si/e. 

The division algorithm: Let a and d be integers with d 
positive. Then there are unique integers q and r with 0 £ 
r < d such that a = dq + r. 

If a and b are positive integers, then ah = gedfa, b) - 
1 ctn(a, b). 

The Euclidean algorithm: for finding greatest common 
divisors (see Algorithm 1 in Section 2.4). 

Let b be a positive integer greater than 1. Then if n is a 
positive integer, it can be expressed uniquely in the form 
n - a t b k + ai t ..ib k 1 +-(• a,b + oq. 

The algorithm for finding the base b expansion of an integer 
(see Algorithm 2 in Section 2.4), 

The conventional algorithms for addition and multiplication 
of integers (given in Section 2.4), 

The greatest common divisor of two integers can be ex¬ 
pressed as a linear combination with integer coefficients 
of these integers. 

If m is a positive integer and gcd(ci, m) = 1 , then a has a 
unique inverse modulo rn. 

The Chinese Remainder Theorem: A system of lin¬ 
ear congruences modulo pairwise relatively prime inte¬ 
gers has a unique solution modulo the product of these 
moduli, 

Fermat's little Theorem: If p is prime and p /a), then 
a'"' 1 2 3 = 1 (mod p ). 


Review Questions 


1. a) Define the term algorithm. 

b) What are the different ways to describe algorithms? 

c) What is the difference between an algorithm for 
Solving a problem and a computer program tliat 
solves this problem? 

2. a) Describe, using English, an algorithm for finding 

the largest, second largest, and third largest integers 
in a list of n integers. 

b) Express this algorithm in pseudocode, 

c) How many comparisons does the algorithm use? 

3. a) Define what the worst-case time complexity, 

average-case time complexity, and best-case time 
complexity (in terms of comparisons) mean for an 
algorithm that finds the smallest integer in a list of 
ft integers. 


h) What are the worst-case, average-case, and best- 
case time complexities, in terms of comparisons, of 
the algorithm that finds the smallest integer in a list 
of n integers by comparing each of the integers with 
the smallest integer found so far? 

4, a) Describe the linear search and binary search algo¬ 

rithm far finding an integer in a list of integers in 
increasing order, 

b) Compare the worst-case time complexities of these 
two algorithms. 

c) Is one of these algorithms always faster than the 
other (measured in terms of comparisons)? 

5, State the Fundamental Theorem of Arithmetic, 

6 , a) Describe a procedure for finding the prime factor¬ 

ization of an integer. 
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b) Use this procedure to find the prime factorization of 
80,707. 

7. a) Define the greatest common divisor of (wo integers. 

b) Describe at least three different ways to find the 
greatest common divisor of two integers. When 
does each method work best? 

c) Find the greatest common divisor of 1,234,567 and 
7,654,321. 

d) Find the greatest common divisor of 2 1 2 3 4 3 s 5 7 * 7 9 ll 
and 2/*2l. 7 6 5 5 7 3 13. 

8 . a) Define what it means for a and b to be congruent 

modulo 7. 

b) Which pairs of the integers — 11 , — S, — 7, — 1,0, 3, 
and 17 are congruent modulo 7? 

c) Show that if a and h are congruent modulo 7, then 
lOfl + 13 and -46 + 20 are also congruent mod¬ 
ulo 7, 

9. Describe a procedure for convening decimal (base 10 ) 
expansions of integers into hexadecimal expansions. 

10, a) How can you fi nd a linear combination (with integer 

coefficients) of two integers that equals their great¬ 
est common divisor? 

b) Express ged(84, 119) as a linear combination of 84 
and 119. 

11. a) What does it mean for a to be an inverse of a mod¬ 

ulo ml 


b) How can you find an inverse of a modulo m when 
m is a positive integer and gcd(u, m) - 1 ? 

e) Find an inverse of 7 modulo 19. 

12. a) How can an inverse of a modulo m be used to 

solve the linear congruence ax — b (mod m) when 
ged(a, m) = 1 ? 

b) Solve the linear congruence 7i = 13 (mod 19). 

13. a) State the Chinese Remainder Theorem, 

b) Find the solutions to the system x = 1 (mod 4), .v = 
2 (mod 5), and x = 3 (mod 7). 

14. Suppose that 2" 1 3 1 (mod n). Is« necessarily prime? 

15. a) What is the difference between a public key and a 

private key cryptosystem? 

b) Explain why using shift ciphers is a private key sys¬ 
tem. 

c) Explain why the RSA cipher system is a public key 

system. 

16. Define the product of two matrices A and B. When is 
this product defined? 

17. a) How many different ways are there to evaluate 

the product A s A : A,Ar by successively multiply¬ 
ing pairs of matrices, when this product is defined? 
b) Suppose that Aj, A 2 , Aj, and A 4 are 10 x 20,20 x 5 , 

5 x K), and 10 x 5 matrices, respectively. How 
should A 1 A 2 A 3 A 4 be computed to use the least 
number of multiplications of entries? 


Supplementary Exercises 


1. a) Describe art algorithm for locating the last occur¬ 

rence of the largest number in a list of integers, 
b) Estimate the number of comparisons used. 

2. a) Describe an algorithm for finding the first and sec¬ 

ond largest elements in a list of integers, 
b) Estimate the number of comparisons used. 

3. at Give an algorithm to determine whether a bit string 

contains a pair of consecutive zeros, 
b) How many comparisons does the algorithm use? 

4. a) Suppose that a list contains integers that are in or¬ 

der of largest to smallest and an integer can appear 
repeatedly in this list. Devise an algorithm that lo¬ 
cates all occurrences of an integer x in the list, 
b) Estimate the number of comparisons used, 

5. Find four numbers that are congruent to 5 modulo 17, 

6 . Show that if a and d arc positive integers, then there are 
integers q and r such that a = dq + r where -d/2 < 
r < dll. 

*1. Show that if ac 3 be (mod m), then a = b[ mod mid) 
where d = gcd(m, c). 

*8. How many zeros are at the end of the binary expansion 

of I OC), n f ? 


9. Use the Euclidean algorithm to find the greatest com¬ 
mon divisor of 10,223 and 33,341. 

19. How many divisions are required to find ged( 144, 233) 
using the Euclidean algorithm? 

11. Find ged(2n -t- 1,3n j- 2), where n is a positive integer. 
(Hint; Else the Euclidean algorithm.) 

12. a) Show that if a and b are positive integers with 

a fc b, then gedf a, b) = a if a = b. gedta. b) = 
2 gcd(o/ 2 , b/2) if a and h are even, ged(a.h) = 
geo-a, 2 . l>) if a is even and b is odd, and gcd(a, b ) -- 
gedffi - 6) if both a and b are odd. 

b) Explain how to use (a) to construct an algorithm for 
computing the greatest common divisor of two pos¬ 
itive integers that uses only comparisons, subtrac¬ 
tions, and shifts of binary expansions, without using 
any divisions. 

c) Find gcd(1202,4848) using this algorithm, 

13. Show that an integer is divisible by 9 if and only if the 
sum of its decimal digits is divisible by 9 . 

14. a) Devise an algorithm for computing x? mod m, 

where x is an integer and m and n arc posrtivc 
integers, using the binary expansion of n. (Hint: 
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Perform successive squarings to obtain * mod m, 
a-’- mod m, a 4 mod m, and so on. Then multiply the 
appropriate powers of the form .r 1 2 3 4 5 ' mod m to obtain 

v" mod m. ) 

b) Estimate the number of multiplications used by this 
algorithm. 

A set of integers is called mutually relatively prime if the 

greatest common divisor of these integers is 1 . 

15. Determine whether the following sets of integers arc 
mutually relatively prime. 

a) 8, 10. 12 

bji 12. 15,25 

c) 15.21,28 

d> 21, 24,28.32 

16. Find a set of four mutually relatively prime integers 
such that no two of them are relatively prime. 

17. a) Suppose that messages are encrypted using the 

function f(p) = [ap + b) mod 26 such that 
gedfs, 26) = I. Determine a function that can be 
used to decry pt messages. 

b> The encrypted version of a message is LJMKG 
MOMXF QEXMW. If it was encrypted using the 
function f(p) = Op + 10 ) mod 26, what was the 
original message? 

18. Show that the system of congruences x = 2 (mod 6 ) 
and \ = 3 (mod 9) has no solutions. 

19. Find till solutions of the system of congruences .< — 
4 (mod 6) and a ^ 13 (mod 15). 

*20, a) Show that the system of congruences x s a, 
(mod iii|j and x - jij (mod mp has a solution 
if and only if gedifoq, wi 2 ) i oi - «;■ 

It) Show that the solution in part (a) is unique modulo 
lemu?!i, (Hi). 


21, Find A” if A is 



22, Show that if A = cl, where c i,s a real number and 1 is 
the n x n identity matrix, then AB ~ BA whenever B 
is an it X rt matrix. 

23, Show' that if A is a 2 X 2 matrix such that AB — BA 
whenever B is a 2 X 2 matrix, then A = cl, where c is 
a real number and I is the 2 x 2 identity matrix. 

An ft x n matrix is called upper triangular if a tJ = 0 

whenever i > /. 

24, From the definition of the matrix product, devise an al¬ 
gorithm for computing the product of two upper tri¬ 
angular matrices that ignores those products in the 
computation that are automatically equal to zero. 

25, Give a pseudocode description of the algorithm in Ex¬ 
ercise 24 for multiplying two upper triangular matrices. 

26, How many multiplications of entries are used by the 
algorithm found in Exercise 25 for multiplying two n x 
n upper triangular matrices? 

27, Show that if A and B are invertible matrices and AB 
exists, then (AB ) -1 = B -1 A *. 

28, What is the best, order to form the product ABCD if A, 
B. C, and D are matrices with dimension! 30 x 10,10 x 
40, 40 x 50, and 50 x 30, respectively? Assume that the 
number of multiplications of entries used to multiply a 
p X q matrix and a?Xr matrix is pqr, 

29, Let A be an n X n mam* and Jet 0 be ihe n x >i matrix 
all of whose entries are zero. Show that the following 
are true. 

a) AGO = ftGA = 0 b) Ay0 = 0 \ A - A 
c) A A 0 = 0 A A = 0 


Computer Projects 

WRITE PROGRAMS WITH THE FOLLOWING INPUT AND OUTPUT. 


1. Given a list of n integers, find the largest integer in the 
list. 

2. Given a list of n integers, find the first and last occur¬ 
rences of the largest integer in the list. 

3. Given a list of n distinct integers, determine the posi¬ 
tion of an integer in the list using a linear search. 

4. Given an ordered list of n distinct integers, determine 
the position of an integer in the listusing abinary search 

5. Given an ordered list of n integers and an integer x, 
find the number of comparisons used to determine the 
position uf an integer in the list using a linear search 
and using a hi nary search. 


6 . Given apositive integer, determine whether it is prime. 

7. Given a message, encrypt this message using the Cae¬ 
sar cipher; and given a message encrypted using the 
Caesar cipher, decrypt this message. 

8. Given two positive integers, find their greatest common 
divisor using the Euclidean algorithm. 

9. Given two positive integers, find their least common 
multiple. 

*10, Given a positive integer, find the prime factorization of 
this integer. 

II. Given a positive integer and a positive integer b great- 
cr than 1 . find the base b expansion of this integer. 
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12. Given a positive integer, find the Cantor expansion of 
this integer (see the preamble to Exercise 32 of Sec¬ 
tion 2.4). 

13. Given a positive integer n, a modulus m, multiplier a, 
increment e, and seed x ti , with 0 < a < m, 0 £ c < 
m, and 0 it r 0 < m, generate the sequence of n pseu¬ 
dorandom numbers using the linear cungruential gen¬ 
erator x n ^i = (fljr, + ci mod m. 

14. Given positive integers a and b. find integers s anti t 
such that sa + sb = ged(a, b). 

15. Given n linear congruences modulo pairwise relatively 
prime moduli, find the simultaneous solution of these 
congruences modulo the product of these moduli. 

16. Given an m x k matrix A and a k x « matrix B, find 
AB. 


17, Given a square matrix A and a positive integer n, find 
A". 

18, Given a square matrix, determine whether it is sym¬ 
metric. 

19, Given an ;t| x hi matrix A, aiwii x n, matrix B, 
an n. x n 4 matnx C, and an in x n 5 matrix D, 
all with integer entries, determine the most effi¬ 
cient order to multiply these matrices (in terms of the 
number of multiplications and additions of integers f 

20, Given two m X n Boolean matrices, find their meet and 
join. 

21, Given an m x k Boolean matrix A and a kx n Boolean 
matrix B, find the Boolean product of A and B. 

22, Given a square Boolean matrix A and a positive integer 
n, find A 1 ” 1 . 


Computations and Explorations 

1 SR A a IMPUTATION A L PROGRAM OR PROGRAMS YOU HAVE WRITTEN TO DO THE FOLLOWING EXERCISES. 


1. Determine whether 2 P - 1 is prime for each of the 
primes not exceeding 100 , 

2. Test a range of large Mcrscnne numbers 2 1 ’ - I to de¬ 
termine whether they are pnme. (You may want to use 
software from the GIMPS project.) 

3. Show that rr + n + 41 is prime for all integers n with 
0 -x n -x 39, but is not prime when n = 40. Is there 
a polynomial in n with integer coefficients and degree 
greater than zero that always takes on a prime value 
when ?r is a positive integer? 

4. Find as many primes of the form tr - i where n is 
a positive integer as you can. It is not known whether 
there are infinitely many such primes. 

5. Find 10 different primes each with 100 digits. 


6 . How many primes are there less than 1,000,000, less 
than 10,000,000. and less than 100,000,000? Can you 
propose an estimate for the number of primes less than 
x where v is a positive integer? 

7. Find a prime factor of each of 10 different 20-digit odd 
integers, selected at random. Keep track of how long it 
takes to find a factor of each of these integers. Do the 
same thing for 10 different 30-digit odd integers. 10 
different 40-digit odd integers, and so on, continuing 
as long as passible. 

8 . Find all pseudoprimes to the base 2, that is, composite 
integers n such that 2"' 1 = 1 (mod n\ where n does 
not exceed 10 , 000 . 


Writing Projects 

RESPOND TO THF FOLLOWING PROJECTS WITH ESSAYS USING OUTSIDE SOURCES. 


Examine the history of the word algorithm and de¬ 
scribe the use of this word in early writings, 

2. Describe what is meant by a parallel algorithm. Explain 
how the pseudocode used m this book can be extended 
to handle parallel algorithms. 

3. Explain how the complexity of parallel algorithms can 
be measured. Give some examples to illustrate this con¬ 
cept. showing how a parallel algorithm can work more 
quickly than one that does not operate in parallel. 

4. Describe the Lucas -Lehmer test for determining 
whether a Mcrscnne number is prime. Discuss the 


progress of the GIMPS project in finding Mcrscnne 
primes using this test. 

5. Explain how probabilistic pnmaiity tests are used in 
practice to produce extremely large numbers that are 
almost certainly prime. Do such tests have any poten¬ 
tial drawbacks? 

6 , A Carmichael number is an integer that is a pseu¬ 
doprime to all bases relatively prime to this integer. 
The question of whether there are infinitely many 
Carmichael numbers was solved recently after be¬ 
ing open tor more than 75 years. Explain what a 
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Carmichael number is, give examples of such numbers, 
and describe the ingredients that went into the proof 
that there are infinitely many such numbers. 

7. Summarize the current status of factoring algorithms 
in terms of their complexity and the size of numbers 
that can currently be factored. When do you think that 
it will be feasible to factor 200-digit numbers? 

8. Describe the algorithms that are actually used in mod¬ 
ern computers to add, subtract, multiply, and divide 
positive integers. 

9. Describe the history' of the Chinese Remainder Theo¬ 
rem. Describe some of the relevant problems posed in 
Chinese and Hindu writings and how the Chinese Re¬ 
mainder Theorem applies to them, 

10. When are the numbers of a sequence truly random 
numbers, and not pseudorandom? What shortcomings 
have been observed in simulations and experiments in 


which pseudorandom numbers have been used? What 
are the properties that pseudorandom numbers can 
have that random numbers should not have? 

11. Describe ho w public key cryptography is being applied. 
Are the ways it is applied secure given the status of fac¬ 
toring algorithms? Will information kept secure using 
public key cryptography become insecure in the future? 

12. Describe how public key cryptography can be used to 
send signed secret messages so that the recipient is rel¬ 
atively sure the message was sent by the person claim¬ 
ing to have sent it. 

13. Show how a congruence can be used to tell the day of 
the week for any given date. 

14. Describe some of the algorithms used to efficiently 
multiply large integers. 

15. Describe some of the algorithms used to efficiently 
multiply large matrices. 
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|o understand written mathematics, we must understand what makes up a correct 


mathematical argument, that is, a proof. To learn mathematics, a person needs 


.M. to construct mathematical arguments and not just read exposition. Obviously, 
this requires an understanding of the techniques used to build proofs. The goals of this 
chapter are to teach what makes up a correct mathematical argument and to give the 
student the necessary tools to construct these arguments. 

Note that the methods we will study for building proofs are also used throughout 
computer science, such as the rules computers use to reason, the techniques used to 
verify that programs are correct, and the rules used for constructing new theorems using 
automated reasoning. 

Many mathematical statements assert that a property is true for all positive integers. 
Examples of such statements are that for every positive integer n: n! ^ tf, tr - n 
is divisible by 3, and the sum of the first n positive integers is n(n - t)/2. A major 
goal of this chapter, and the book, is to give the student a thorough understanding of 
mathematical induction, which is used to prove results of this kind. 

In previous chapters we explicitly defined sets, sequences, and functions. That is, 
we described sets by listing their elements or by giving some property that characterizes 
these elements. We gave formulae for the terms of sequences and the values of functions. 
There i s another important way to define such objects, based on mathematical induction. 
To define sequences and functions, some initial terms are specified, and a rule is given 
for finding subsequent values from values already known. For instance, we can define 

the sequence {2"} by specifying that a\ ~ 2 and that n H+ ] — 2ti n for n - 1,2, 3. 

Sets can be defined by listing some of their elements and giving rules for constructing 
elements from those already known to be in the set. Such definitions, called recursive 
definitions, are used throughout discrete mathematics and computer science. 

When a procedure is specified for solving a problem, this procedure always solves 
the problem correctly. Just testing to see that the correct result is obtained for a set of input 
values does not show that the procedure always works correctly. Thecorrectnessofaproee- 
durecanbeguaranteedonly by proving that it always yields theeorrectresult.Thefinalsec- 
tionof this chapter containsan introduction to thetechniques of program verification. This 
is a formal technique to verify that procedures are correct. Program verification serves as 
the basi s for attempts underway to prove in a mechanical fashion that programs are correct. 


3.1 


Methods of Proof 


INTRODUCTION 


Two important questions that arise in the study of mathematics are: (1) When is a math¬ 
ematical argument correct? (2) What methods can be used to construct mathematical 
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arguments? This section helps answer these questions by describing various forms of 
correct and incorrect mathematical arguments. 

A theorem is a statement that can be shown to be true. We demonstrate that a 
theorem is true with a sequence of statements that form an argument, called a proof. 
To construct proofs, methods are needed to derive new statements from old ones. The 
statements used in a proof can include axioms or postulates, which are the underlying 
assumptions about mathematical structures, the hypotheses of the theorem to be proved, 
and previously proved theorems. The rules of inference, which are the means used to 
draw conclusions from other assertions, tie together the steps of a proof. 

In this section rules of inference will be discussed. This will help clarify what 
makes up a correct proof. Some common forms of incorrect reasoning, called fallacies, 
will also be described. Then various methods commonly used to prove theorems will 
be introduced. 


Remark: The terms lemma and corollary are used for certain types of theorems. A 
Jemma (plural le mm as or lemmata) is a simple theorem used in the proof of other 
theorems. (For instance, Lemma 1 in Section 2.4 was used to prove the theorem that 
the Euclidean algorithm produces the greatest common divisor of two integers.) Com¬ 
plicated proofs are usually easier to understand when they are proved using a series of 
lemmas, where each lemma is proved individually. A corollary is a proposition that 
can be established directly from a theorem that has been proved. A conjecture is a 
statement whose truth value is unknown. When a proof of a conjecture is found, the 
conjecture becomes a theorem. Many times conjectures are shown to be false, so they 
are not theorems. 


The methods of proof discussed in this chapter are i mportant not only because they 
are used to prove mathematical theorems, but also for their many applications to com¬ 
puter science. These applications include verifying that computer programs are correct, 
establishing that operating systems are secure, making inferences in the area of artifi¬ 
cial intelligence, and so on. Consequently, understanding the techniques used in proofs 
is essential both in mathematics and in computer science. 


RULES OF INFERENCE 

We will now introduce rules of inference for propositional logic. These rules provide the 
justification of the steps used to show that a conclusion follows logically from a set of 
hypotheses. The tautology {p A (p — q)) —> q is the basis of the rule of inference called 
modus ponens, or the law of detachment. This tautology is written in the following way: 

P 

p q 

:,q 


Using this notation, the hypotheses are written in a column and the conclusion below a 
bar. (The symbol.-. denotes “therefore”) Modus ponens states that if both an implication 
and its hypothesis are know n to be true, then the conclusion of this implication is true. 


EXAMPLE 1 Suppose that the implication “if it snows today, then we will go skiing” and its hy¬ 

pothesis, “it is snowing today,” are true. Then, by modus ponens, it follows that the 
conclusion of the implication, “we will go skiing,” is true. ■ 
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EXAMPLE 2 


EXAMPLE 3 


TABLE I Rules of Inference. 

Rule of inference 

Tautology 

Name. 

P 

P V <7 

p -* (pyq) 

Addition 

pp'q 

P 

ip A q)-* p 

Simplification 

P 

q 

LiO -'-'.vD (pP.q) 

Conduction 

P A q 



P 

P_~*Jj 

■q 

[p A ip — $)] q 

Modus ponens 

q 

-,p 

[-k qAip • q) ] — -ip 

Modus tolfcns 

p q 

4 ~~ r 

■ ■P^r 

i(P q)A(q -> yj] (p — r) 

1 

Hypothetical syllogism 

pyq 

ZP _ 

" q 

f(p V?) A -ip] q 

_ 

Disjunctive syllogism 


The implication “if n is divisible by 3, then n 2 is divisible by 9," is true. Consequently, 
if n is divisible by 3, then by modus ponens, it follows that rr is divisible by 9. ■ 


Table l lists some important rules of inference. The verifications of these rules of 
inference can be found as exercises in Section 1.2. Here are some examples of argu¬ 
ments using these rules of inference. 


State which rule of inference is the basis of the following argument: “It is below freezing 
now. Therefore, it is either below freezing or raining now,” 

Solution: Let p be the proposition “It is below freezing now” and q the proposition “It 
is raining now.” Then this argument is of the form 

P _ 

:.py'q 

This is an argument that uses the addition rule, ■ 
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EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


State which rule of inference is the basis of the following argument: “Ft is below freezing 
and raining now. Therefore, it is below freezing now,” 

Solution: Let p be the proposition “It is below freezing now,” and let q be the proposi¬ 
tion “It is raining now.” This argument is of the form 

P A g 

:.p 

This argument uses the simplification rule. ■ 


State which rule of inference is used in the argument: 

If it rains today, then we wall not have a barbecue today. If we do not have a barbecue 
today, then we w ill have a barbecue tomorrow. Therefore, if it rains today, then we 
will have a barbecue tomorrow. 

Solution: Let p be the proposition “It is raining today,” let q be the proposition “We 
will not have a barbecue today,” and let r be the proposition “We will have a barbecue 
tomorrow.” Then this argument is of the form 

P “* ? 

<i r 

:,p r 

Hence, this argument is a hypothetical syllogism. ■ 


An argument is called valid if whenever ail the hypotheses are true, the conclu¬ 
sion is also true. Consequently, showing that q logically follows from the hypotheses 
P\. P2 . p„ is the same as showing that the implication 

if 1 : • /'. ''•• •' /V q 

is true. When all propositions used in a valid argument are true, it leads to a correct 
conclusion. However, a valid argument can lead to an incorrect conclusion if one or 
more false propositions are used within the argument. For example, 

“If 101 is divisible by 3, then 101 2 is divisible by 9. 101 is divisible by 3, Conse¬ 
quently, 101 2 is divisible by 9.” 

is a valid argument based on modus ponens. However, the conclusion of this argument 
is false, since 9 does not divide 101 2 = 10,201. The false proposition “101 is divisible 
by T has been used in the argument, which means that the conclusion of the argument 
may be false. 

When there are many premises, several rules of inference are often needed to show 
that an argument is valid. This is illustrated by the following examples, where the steps 
of arguments arc displayed step by step, with the reason for each step explicitly stated. 
These examples also show how arguments in English can be analyzed using rules of 
inference. 


Show that the hypotheses “It is not sunny this afternoon and it is colder than yesterday.” 
We will go swimming only if it is sunny,” “If wc do not go swimming, then we will 
take a canoe trip,” and “If we take a canoe trip, then we will be home by sunset” lead 
to the conclusion “We will be home by sunset.” 
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Solution: Let p be the proposition “It is sunny this afternoon,” q the proposition “It 
is colder than yesterday,” r the proposition “We will go swimming,” s the proposi¬ 
tion “We will take a canoe trip,” and t the proposition “We will be home by sunset,” 
Then the hypotheses become -ipf\q r r-+p,-tr—* s. and s — t. The conclusion is 
simply t. 

We construct an argument to show that our hypotheses lead to the desired conclu¬ 
sion as follows. 


Step Reason 

1 -i p A q Hypothesis 

2. ip Simplification using Step 1 

3. r p Hypothesis 

4. -if Modus tollens using Steps 2 and 3 

5. r Hypothesis 

6. s Modus ponens using Steps 4 and 5 

7. i —* t Hypothesis 

8. t Modus ponens using Steps 6 and 7 ■ 


Show that the hypotheses “If you send me an e-mail message, then I will finish writ¬ 
ing the program,” “If you do not send me an e-mail message, then I will go to sleep 
early,” and “If I go to sleep early, then I will wake up feeling refreshed” lead to 
the conclusion “If I do not finish writing the program, then 1 will wake up feeling 
refreshed.” 

Solution: Let p be the proposition “You send me an c-mail message,” q the proposition 
“1 will finish writing the program,” r the proposition “I will go to sleep early,” and i 
the proposition “I will wake up feeling refreshed.” Then the hypotheses are p q, 
-ip and r —* s. The desired conclusion is -i q s. 

The following argument shows that our hypotheses lead to the desired conclusion. 

Reason 
Hypothesis 

Contrapositive of Step 1 
Hypothesis 

Hypothetical syllogism using Steps 2 and 3 
Hypothesis 

Hypothetical syllogism using Steps 4 and 5 ■ 


Step 

L p — q 

2. -i r j —* -i p 

3. -ip -* r 

4. -iq r 

5. r —> s 

6. -if/ r 


FALLACIES 

Several common fallacies arise in incorrect arguments. These fallacies resemble rules 
of inference but are based oil contingencies rather than tautologies. These are discussed 
here to show the distinction between correct and incorrect reasoning. 
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The proposition [ip — 1 * q) A <?] — p is not a tautology, since it is false when p 
is false and q is true, However, there are many incorrect arguments that treat this as a 
tautology. This type of incorrect reasoning is called the fallacy of affirming the con¬ 
clusion. 


Is the following argument valid? 

If you do every problem in this book, then you will leam discrete mathematics. You 
learned discrete mathematics. 

Therefore, you did every problem in this book. 

Solution: Let p be the proposition “You did every problem in this book.” Let q be the 
proposition “You learned discrete mathematics.” Then this argument is of the form: if 
P —> q and q, then p. This is an example of an incorrect argument using the fallacy 
of affirming the conclusion. Indeed, it is possible for you to leam discrete mathematics 
in some way other than by doing every problem in this book, (You may leam discrete 
mathematics by reading, listening to lectures, doing some but not all the problems in 
this book, and so on.) ■ 


Let p be the proposition “« - 1 (mod 3).” and let q be the proposition “>r - 1 (mod 
3).” The implication p -»■ q, which is “if n = 1 (mod 3), then n 1 = 1 (mod 3),'* is true, 
if q is true, so that ir - 1 (mod 3), does it follow that p is true, namely, that n - 1 
(mod 3)? 

Solution; It would be incorrect to conclude that p is true, since it is possible that n ^ 2 
(mod 3). If the incorrect conclusion that p is true is made, this would be an example of 
the fallacy of affirming the conclusion. ■ 


The proposition f(p —> i?)A-i p] -* - q is not a tautology, since it is false when p is 
false and q is true. Many incorrect arguments use this incorrectly as a rule of inference. 
This type of incorrect reasoning is called the fallacy of denying the hypothesis. 


Let p and q be as in Example 8. U the implication p ^ q is true, and -ip is true, is it 
correct to conclude that -iq is true? In other 'words, is it correct to assume that you did 
not leam discrete mathematics if you did not do every problem in the book, ass umin g 
that if you do every problem in this book, then you will leam discrete mathematics? 


Solution: It is possible that you learned discrete mathematics even if you did not do 
every problem in this book. This incorrect argument is of the form p —* q and -ip 
imply q, which is an. example of the fallacy of denying the hypothesis. ■ 


Let p and q be as in Example 9. Is it correct to assume that if -ip is true, then ->q is 
true, using the fact that p —> q is true? In other words, is it correct to conclude that 
rr ^ 1 (mod 3) if n ^ 1 (mod 3), using the implication: if n ~ 1 (mod 3), then n 2 - 1 
(mod 3)? 
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Solution; It is incorrect to conclude that rr f I (mod 3) if n £ 1 (mod 3), since n 2 - 1 
(mod 3} when n = 2 {mod 3), This incorrect argument is another example of the fallacy 
of denying the hypothesis. ■ 


Many incorrect arguments are based on a fallacy called begging the question. This 
fallacy occurs when one or more steps of a proof are based on the truth of the statement 
being proved. In other words, this fallacy arises when a statement is proved using itself, 
or a statement equivalent to it. That is why this fallacy is also called circular reasoning. 


EXAMPLE 12 Is the following argument correct? It supposedly shows that n is an even integer when¬ 

ever n 2 is an even integer. 

Suppose that n 2 is even. Then rr - 2k for some integer k. Let n - 2/ for some 
integer l. This shows that n is even. 

Solution: Tills argument is incorrect. The statement "let n = 21 for some integer f’ 
occurs in the proof No argument has been given to show that it is true. This is circular 
reasoning because this statement is equivalent to the statement being proved, namely, 
'Vi is even.” Of course, the result itself is correct; only the method of proof is wrong. ■ 

RULES OF INFERENCE FOR QUANTIFIED STATEMENTS 

We discussed rules of inference for propositions. We will now describe some important 
rules of inference for statements involving quantifiers. These rules of inference are used 
extensively in mathematical arguments, often without being explicitly mentioned. 

Universal instantiation is the rule of inference used to conclude that Pic) is true, 
where c is a particular member of the universe of discourse, given the premise VxP(x). 
Universal instantiation is used when we conclude from the statement “All women are 
wise” that “Lisa is wise,” where Lisa is a member of the universe of discourse of ail 
women. 

Universal generalization is the rule of inference which states that VxPU) is true, 
given the premise that P(c) is true for all elements c in the universe of discourse. Uni¬ 
versal generalization is used when we show that VxP(x) is true by taking an arbitrary’ 
element c from the universe of discourse and showing that P(c) is true. The element c 
that we select must be an arbitrary, and not a specific, element of the universe of dis¬ 
course. Universal generalization is used implicitly in many proofs in mathematics and 
is seldom mentioned explicitly. 

Existential instantiation is the rule which allows us to conclude that there is an 
element c in the universe of discourse for which P(c) is true if we know that 3xP(x) is 
true. We cannot select an arbitrary value of c here, but rather it must be a c for which 
P[c) is true. Usually we have no knowledge of what c is, only that it exists. Since it 
exists, we may give it a name (c) and continue our argument. 

Existential generalization is the rule of inference winch is used to conclude that 
3xP(x) is true when a particular element c with P(c ) true is known. That is, if we know 
one element c in the universe of discourse for which P(e) is true, then we know that 
3.vPu) is true. 

We summarize these rules of inference in Table 2. 

We will illustrate how one of these rules of inference for quantified statements is 
used in the following example. 
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TABLE 2 Rules of Inference for Quantified Statements, 

V Is the Universe of Discourse, 

Rule of Inference 

Name 

Va'P(x) 
p(c) if c 0 V 

Universal instantiation 

P(c) for an arbitrary c £ U 
'"VxA’O 

Universal generalization 

3-r P(x) 

.. Fit) for some eJemem c e U 

Existential instantiation 

P(i ) for some element c e U 
3xP(x) 

Existential generalization 


Show that the premises “Everyone in this discrete mathematics class has taken a course 
in computer science” and “Marla is a student in this class” imply the conclusion “Marla 
has taken a course in computer science.” 

Solution: Let DU) denote “v is in this discrete mathematics class,” and let Cu) denote 
“x has taken a course in computer science,” Then the premises are Vx(D(x) —* C(x)) 
and £)(Marla). The conclusion is CfMaria). 

The following steps can be used to establish the conclusion from the premises. 


Step 


Reason 

1. Vx(D(x) - 

■ C(JT» 

Premise 

2. D(Marla) - 

• ('Mai la: 

Universal instantiation using Step 1 

1 D(Marla) 


Premise 

4. CtMarla) 


Modus ponens using Steps 2 and 3 


Remark: Mathematical arguments often include steps where both a rule of inference 
for propositions and a rule of inference for quantifiers are used. For example, universal 
instantiation and modus poriens are often used together. When these rules of inference 
are combined, the hypothesis Vx(P(x) — Q(x)) and Pic), where c is a member of the 
universe of discourse, show that the conclusion Q(c) is true. 

Remark: Many theorems in mathematics state that a property holds for all elements 
in a particular set, such as the set of integers or the set of real numbers. Although the 
precise statement of such theorems needs to include a universal quantifier, the standard 
convention in mathematics is to omit it. For example, the statement “If the integer n 
is divisible by 3, then n 2 is divisible by 9” really means “For all integers n, if n is 
divisible by 3, then rr is divisible by 9.” Similarly, the statement “If x > y, where 
.t and y are positive retd numbers, then x 2 > v 2 ” really means “For all positive real 
numbers x and y, if x > y, then x 1 > y 2 .” Furthermore, when theorems of this type 
are proved, the law of universal generalization is often used without explicit mention. 
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The first step of the proof usually involves selecting a general element of the universe 
of discourse. Subsequent steps show that this element has the property in question. 
Universal generalization implies that the theorem holds for all members of the universe 
of discourse. 

In our subsequent discussions, we will follow the usual conventions and not explic¬ 
itly mention the use of universal quantification and universal generalization. However, 
you should always understand when this rule of inference is being implicitly applied. 


METHODS OF PROVING THEOREMS 


We proved several theorems in Chapters 1 and 2. Let us now be more explicit about the 
methodology of constructing proofs. We will describe how different types of statements 
are proved. 

Because many theorems are implications, the techniques for proving implications 
are important. Recall that p -*■ q is true unless p is true but q is false. Note that when 
the statement p q is proved, it need only be shown that q is true if p is true; it is no! 
usually the case that q is proved to be true. The following discussion will give the most 
common techniques for proving implications. 

The implication p — q can be proved by showing that if p is true, then q must also 
be true. This show's that the combination p true and q false never occurs. A proof of this 
kind is called a direct proof. To cany out such a proof, assume that p is true and use 
rules of inference and theorems already proved to show that q must also be true. 


Give a direct proof of the theorem "If n is odd, then rr is odd.” 

Solution: Assume that the hypothesis of this implication is true, namely, suppose that 
n is odd. Then n = 2k + 1, where k is an integer. It follows that n 2 (2k + l) 2 - 

4k 2 + 4k + 1 ~ 2(2k 2 + 2k) + 1. Therefore, n 2 is odd (it is 1 more than twice an 
integer). ■ 


Since the implication p ^ q is equivalent to its contrapositive, iq -tp, the 
implication p -*■ q can be proved by showing that its contrapositive, ~iq — -i p, is 
true. This related implication is usually proved directly, but any proof technique can be 
used. An argument of this type is called an indirect proof. 


Give an indirect proof of the theorem “If 3ri + 2 is odd, then n is odd.” 

Solution: Assume that the conclusion of this implication is false; namely, assume that 
n is even. Then n - 2k for some integer k, It follows that 3n + 2 - 3(2 k) + 2 = 
+ 2 = 2(3fc + 1), so 3n + 2 is even (since it is a multiple of 2). Since the negation 
of the conclusion of the implication implies that the hypothesis is false, the original 
implication is true. g 


Suppose that the hypothesis p of an implication p q is false. Then the implica¬ 
tion p - 1 q is true, because the statement has the form E -- T or F • F, and hence 
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EXAMPLE 18 


is true. Consequently, if it can be shown that p is false, then a proof, called a vacu¬ 
ous proof, of the implication p -* q can be given. Vacuous proofs are often used to 
establish special cases of theorems that state that an implication is true for all positive 
integers | i.e., a theorem of the kind 'in P(n) where Pin) is a propositional function]. 
Proof techniques for theorems of this kind will be discussed in Section 3.2. 


Show that the proposition P(0) is true w here Pi n) is the propositional function "If n > 1, 
then tr > n." 

Solution: Note that the proposition PfO) is the implication “If 0 > I, then 0 2 > 0." 
Since the hypothesis 0 > 1 is false, the implication P(0) is automatically true. ■ 


Remark: The fact that the conclusion of this implication, O 2 > 0, is false is irrelevant 
to the truth value of the implication, because an implication with a false hypothesis is 
guaranteed to be true. 

Suppose that the conclusion q of an implication p q is true. Then p —* q is 
true, since the statement has the form T —» T or F — 1 ■ T, which are true. Hence, if it 
can be shown that q is true, then a proof, called a trivial proof, oi p q can be given. 
Trivial proofs are often important when special cases of theorems are proved (see the 
discussion of proof by cases) and in mathematical induction, which is a proof technique 
discussed in Section 3,2. 


Let Pi n) be the proposition "If a and b are positive integers with a 2 b. then o n 2 h n P 
Show that the proposition P(0) is true. 

Solution: The proposition P(O) is “If a 2 b, then a 0 a b 0 .” Since a 0 = b G - 1, the 
conclusion of P(0) is true. Hence, P(0) is true. This is an example of a trivial proof. 
Note that the hypothesis, which is the statement "a > b" was not needed in this proof. 


Suppose that a contradiction q can be found so that -i p -»■ q is true, that is, -i p — 
F is true. Then the proposition -ip must be false. Consequently, p must be true. This 
technique can be used when a contradiction, such as r A -tr, can be found so that it is 
possible to show that the implication -i p -* (r A r \is true. An argument of this type 
is called a proof by contradiction. 


Prove that J2 is irrational by giving a proof by contradiction. 

Solution: Let p be the proposition “ v '2 is irrational.” Suppose that p is true. Then 
J2 is rational. We will show that this leads to a contradiction, Under the assumption 
that v'2 is rational, there exist integers a and b with y'2 = alb, where a and b have 
no common factors (so that the fraction alb is in lowest terms). Since v"2 = alb, when 
both sides of this equation are squared, it follows that 

2 - trilr. 
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Hence, 

2b 2 = « 2 . 

This means that a 1 is even, implying that a is even. Furthermore, since a is even, a - 2c 
for some integer e. Thus 

lb 1 = 4c 2 , 
so 

b- = 2c 2 . 

This means that b 2 is even. Hence, b must be even as well. 

It has been shown that -i p implies that J2 = aib, where a and b have rto common 
factors, and 2 divides a and b. This is a contradiction since we have shown that -r p 
implies both r and -i r where r is the statement that a and b are integers with no common 
factors. Hence. ->p is false, so that p: “y 2 is irrational" is true. ■ 


An indirect proof of an implication can be rewritten: as a proof by contradiction. 
In an indirect proof we show that p —> q is true by using a direct proof to show that 
-■if/ —i p is true. That is, in an indirect proof of p —* q we assume that -iq is true 

and show that -i p must also be true. To rewrite an indirect proof of p —* q as a proof 
by contradiction, we suppose that both p and -it/ are true. Then we use the steps from 
the direct proof of -i q —>■ -i p to show that -i p must also be true. This leads to the 
contradiction p A p. completing the proof by contradiction. Example 19 illustrates 
how an indirect proof of an implication can be rewritten as a proof by contradiction. 


Give a proof by contradiction of the theorem “If 3 it + 2 is odd, then n is odd." 

Solution: We assume that 3 h+ 2 is odd and that n is not odd, so that n is even. Following 
the same steps as in the solution of Example 15 (an indirect proof of this theorem), we 
can show that if n is even, then in + 2 is even. This contradicts the assumption that 
3tt + 2 is odd. completing the proof, ■ 


To prove an implication of the form 

(Pi VP2 V "■ VP«) “* <? 
the tautology 

Kpi v in V'" V Pn) f I ” Kpi q) A (pi q ) A - ■ ■ A (p„ —■ t/t] 

can be used as a rule of inference. This shows that the original implication with a hy¬ 
pothesis made up of a disjunction of the propositions pi, P 2 , ■■■, pa can be proved by 
proving each of the n implications p-, —* q,i - 1,2. n, individually. Such an argu¬ 

ment is called a proof by cases. Sometimes to prove that an implication p — qis true, 
it is convenient to use a disjunction p\ \J pj \J ■ ■ ■ \j p„ instead of p as the hypothesis of 
the implication, where p and pj V P2 V' • * V P™ are equivalent. Consider the following 
example. 


Prove the implication “If n is an integer not divisible by 3, then n 2 = l (mod 3).” 
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Solution: Let p be the proposition “n is not divisible by 3,” and let a be the proposition 
“n 2 = 1 (mod 3).” Then p is equivalent to p\ \J p 2 where pi is “n = 1 (mod 3)” and 
P2 is “n ~ 2 (mod 3).” Hence, to show that p q it can be shown that p\ quad 
p 2 —» q. It is easy to give direct proofs of these two implications. 

First, suppose that p { is true. Then n ® I (mod 3), so that n - 3 k + 1 for some 
integer k. Thus, 

n 1 = 9k 2 + 6k + 1 = 3(3 E 2 + 2k) + I. 

It follows that >r = 1 (mod 3). Hence, the implication p, q is true. Next, suppose 
that pi is true. Then n = 2 (mod 3), so that n - Ik + 2 for some integer k. Thus, 

n ? - 9 k 2 + 12fc + 4= 20k 1 + 4 k + 5) + 1. 

Hence, n 2 - 1 (mod 3), so the implication p 2 —* # is true. 

Since it has been shown that both p { q and pi q are true, it c an be concluded 
that (p\ V Pi) q is true. Moreover, since p is equivalent to p t V Pi, it follows that 
p -* q is true. ™ 


To prove a theorem that is an equivalence, that is, one that is a statement of the 
form p ' f — q where p and q are propositions, the tautology 

(p ' - q) • • i.-/:• A Uf — p)\ 

can be used. That j>, the proposition "p if and only if q” can be proved if both the 
implications “if p, then q" and “if q, then p" arc proved. 


Prove the theorem “The integer n is odd if and only if n- is odd.” 

Solution: This theorem has the form “p if and only if q," where p is “n is odd” and 
q is “n 2 is odd.” To prove this theorem, we need to show that p q and q —» p are 
tme. 

We have already shown (itt Example 14) that p —*■ q is tme. We will use an indirect 
proof to prove that q —*■ p. Assume that its conclusion is false, namely, that n is even. 
Then n - 2k for some integer k. Then rr = 4Jfc 2 = 2 (2k 1 ), so rr is even (since it is a 
multiple of 2). This completes the indirect proof of q —- p. 

Since wc have shown that both p —> q and q — p are tme, we have shown that 
the theorem is true. ■ 


Sometimes a theorem states that several propositions are equivalent. Such a theo¬ 
rem states that propositions p ir p 2 , jq. p„ are equivalent. This can be written as 

Pi Pn. 

which states that all n propositions have the same truth values. One way to prove these 
mutually equivalent is to use the tautology 

fhi — p 2 w ■ ■ ■ ^ Pn \ [{pi — p 2 ) A (p 2 -> p 3 > A ■ ■ ■ A Cp„ — p!)]. 

I’his shows that if the implications pi —» pi, pi —*■ Ph _ Pn -*• Pi can be shown to 

be true, then the propositions p ;. p 2 . p n are all equivalent. 
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Prove that when n is an integer, the following three statements are equivalent. 
pi : n mod 3 = 1 or n mod 3-2 
pp n is not divisible by 3 
pp. /!'■ - 1 (mod 3) 

Solution; To show that the statements are equi valent, we can prove that the implications 
Pi -= pi, pi —* pi, and pi — 11 pi are true. 

We will use a direct proof to show that /y p > is true. Assume that n mod 3 = 1 
or 2. By the division algorithm, n - 3q + r where 0 £ r < 3. By the definition of mod, 
we have r - n mod 3. Since n is divisible by 3 if and only if r — 0, the assumption 
that n mod 3 = 1 or 2 implies that n is not divisible by 3. This completes the proof that 
Pi Pi is true. 

We have already shown that p> —> pi is true in Example 20. 

We will use an indirect proof to show that pi ~ p\ is true. We assume that the 
conclusion of this implication is false, namely, that n mod 3 is neither 1 nor 2, Since 
n mod 3 equals 0, 1, or 2, we see that n mod 3 = 0. This means that 3 j n, so that 
n = 3k for some integer k. This implies that n 2 - 9 k 2 - 3(3.tr), which shows that 
if = () (mod 3), so that p\ is false. This completes the indirect proof that pi — pj, 
and it also completes the proof of the theorem. I 

THEOREMS AND QUANTIFIERS 


Many theorems are stated as propositions that involve quantifiers. A variety of methods 
are used to prove theorems that are quantifications. We will describe some of the most 
important of these here. 

Many theorems are assertions that objects of a particular type exist, A theorem of 
this type is a proposition of the form 3.r P{x), where A is a predicate. A proof of a 
proposition of the form 3.r P\x) is called an existence proof. There are several ways to 
prove a theorem of this type. Sometimes an existence proof of 3,r P{x) can be given by 
finding an element a such that P{a) is true. Such an existence proof is called construc¬ 
tive. It is also possible to give an existence proof that is nonconstructive; that is, we 
do not find an dement a such that P(a) is true, but rather prove that 3 i P(x) is true in 
some other way. One common method of giving a nonconstructive existence proof is to 
use proof bv contradiction and show 1 that the negation of the existential quantification 
implies a contradiction. The concept of a constructive existence proof is illustrated by 
the following example. 


A Constructive Existence Proof Show that there are n consecutive composite positive 
integers for every positive integer n. Note that this asks for proof of the quantification: 
'in 3x(x + i is composite fori - 1,2,.... n). 

Solution: Let 

x = (n + 1)! + 1. 

Consider the integers 

x + 1, x + 2.jt + n . 
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Note that i + I divides x+i = {n+ 1)! + (( + l)fori = 1,2. n. Hence, n consecutive 

composite positive integers have been given. Note that in the solution a number x such 
that x f / is composite for i - 1,2,... n has been produced. Hence, this is an example 
of a constructive existence proof. ■ 

Remark; The proof in Example 23 can be found in the works of the ancient Greek 
mathematician Euclid, 

An example of a nonconstructive existence proof is given next. 


A Nonconstructive Existence Proof Show that for every positive integer n there is a 
prime greater than n, This problem asks for a proof of an existential quantification, 
namely, 3x Qix), where Q(x) is the proposition “,t is prime and x is greater than n,” and 
the universe of discourse is the set of positive integers. 

Solution; Let n be a positive integer. To show that there is a prime greater than n. 
consider the integer n! + 1, Since even, 1 integer has a prime factor, there is at least one 
prime dividing n\ + L (One possibility is that n! + 1 is already prime.) Note that when 
n! + 1 is divided by an integer less than or equal to n, the remainder equals 1. Hence, 
any prime factor of this integer must be greater than n, This proves the result. This 
argument is a nonconstructive existence proof because a prime larger than n has not 
been produced. It has simply been shown that one must exist. ■ 


Suppose a statement of the form Vx P(x) is false. How can we show this? Recall 
that the propositions -iVa'P(x) and 3 jt —ijP(jt) are equivalent. This means that if we 
find an element a such that P(a) is false, then we have shown that 3t -i Fix) is true, 
which means that V.r P(x) is false. An dement a for which P(a) is false is called a 
counterexample. Note that only one counterexample needs to be found to show that 
V i P(x) is false. 


Show that the assertion “All primes arc odd” is false. 

Solution: The statement “All primes are odd" is a universal quantification, namely, 
VxO(x), 

where 0(a) is the proposition “a is odd,” and the universe of discourse is the set of 
primes. Note that a = 2 is a counterexample, since 2 is a prime number that is even. 
Hence, the statement "All prime numbers are odd" is false. ■ 


It is a common mistake to assume that one or more examples establish the truth of 
a statement. No matter how many examples there are where P(.x) is true, the universal 
quantification VxP(x) may still he false. Consider the following example. 


Is w- it - 41 prime for all nonnegative integers «? That is, is the statement 'inP{n) 
a theorem, where Fin) is the statement - n + 41 is prime” and the universe of 
discourse is the set of nonnegative integers? 
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Solution: To determine whether n L - ti -r 41 is prime for all nonnegative integers, we 
might begin by examining whether it is prime for the smallest nonnegative integers. 
We find that rr - n + 41 is prime for all nonnegative integers not exceeding 40 (as the 
reader can verify). However, if we decided this was enough cheeking, we would come 
to the wrong conclusion. It is not true that n* — n + 41 is prime for all nonnegative 
integers. When n = 41, it is composite (as the reader should verify). ■ 


Example 26 helps illustrate the crucial point that a statement may not be true, even 
though there are many examples for which it is true. 


THE HALTING PROBLEM 


We will now describe a proof of one of the most famous theorems in computer science. 
We will show' that there is a problem which cannot be solved using any procedure. That 
is. we will show there tire unsolvable problems, as was mentioned in Section 2.2. The 
Wtf? problem we will study is the halting problem. It asks whether there is a procedure 
that does the following: It takes as input a computer program and input to (he program 
and determines whether the program will eventually stop when ran with this input. It 
would be convenient to have such a procedure, if it existed. Certainly being able to 
test whether a program entered into an infinite loop would be helpful when writing and 
debugging programs. However, in 1936 Alan Turing showed that no such procedure 
exists (see his biography in Section 10.4). 

Before we present a proof that the halting problem is unsolvable, first note that we 
cannot simply run a program and observe what it does to determine whether it termi¬ 
nates when run with the given input. If the program halts, we have our answer, but if 
it is still running after any fixed length of time has elapsed, we do not know whether it 
will never halt or we just did not wait long enough for it to terminate. After all, it is not 
hard to design a program that will stop only after more than a billion years has elapsed. 

We will describe Turing’s proof that the halting problem is unsolvable; it is a proof 
by contradiction. (The reader should note that our proof is not completely rigorous, 
since we have not explicitly defined what a procedure is. To remedy this, the concept 
of a Turing machine is needed. This concept is introduced in Section 10.5.) 

Proof: Assume there is a solution to the halting problem, a procedure called H{P, I). 
The procedure HiP. /) takes two inputs, one a program P and the other /, an input to the 
program P. H(P, /) generates the string “halt” as output if H determines that P stops 
when given /as input. Otherwise, HiP. 1) generates the string “loops forever” as output. 
We will now derive a contradiction. 

When a procedure is coded, it is expressed as a string of characters; this siring can 
be interpreted as a sequence of bits, This means that a program itself can be used as 
data. Therefore a program can be thought of as input to another program, or even itself. 
Hence, H can take a program P as both of its inputs, which are a program and input to 
this program. H should be able to determine if P will halt when it is given a copy of 
itself as input. 

To show that no procedure H exists wriich solves the halting problem, we construct 
a simple procedure K(P), which works as follows, making use of the output HiP, P). If 
the output of H ( P, P] is “loops forever,” which means that P loops forever when given a 
copy of itself as input, then K[P) halts. If the Output of Hi P, P) is “halt.” which means 
that P halls when given a copy of itself as input, then KIP) loops forever. That is. K ( P ) 
does the opposite of what the output of H(P. P) specifies. (See Figure !.} 
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FIGURE 1 


Now suppose we provide K as input to K. We note that if the output of H(K, K) is 
“loops forever," then by the definition of K we see that K(K) halts. Otherwise, if the 
output of H(K. K) is "halt," then by the definition of K we see that K(K) loops forever, 
in violation of what H tells us. In both eases, we have a contradiction. 

Thus, H cannot always give the correct answers. Consequently, there is no proce¬ 
dure that solves the halting problem. □ 

SOME COMMENTS ON PROOFS 


We described a variety of methods for proving theorems. Observe that no algorithm for 
proving theorems has been given here. Such a procedure does not exist. 

There are many theorems whose proofs are easy to find by directly working through 
the hypotheses and definitions of the terms in the theorem. However, it is often difficult 
to prove a theorem without resorting to a clever use of an indirect proof, a proof by 
contradiction, or some other proof technique. Constructing proofs is an art that can be 
learned only by trying various lines of attack. 

Moreover, many statements that appear to be theorems have resisted the persistent 
efforts of mathematicians for hundreds of years. For instance, as simple a statement 
as "every even positive integer greater than 4 is the sum of two primes” has not yet 
been shown to be true, and no counterexample has been found, although it has been 
■ verified for all even positive integers up to tO 14 . This statement is known as Goldbach’s 
conjecture and is one of many assertions in mathematics with a truth value that is 
unknown. 


Exercises 

1. What rule til' inference is used in each of the following 
arguments ,J 

a) Alice is a mathematics major, Therefore, Alice is 
either a mathematics major or a computer science 
major, 


b) Jerry is a mathematics major find a computer sci¬ 
ence major. Therefore, Jerry is a mathematics 
major, 

e) If it is rainy, then the pool w ill be dosed. It is rainy. 
Therefore, die pool is closed. 


Christian GoMbach (1690-1764j. Christian Goldbach was bom in Konigsberg, Prussia, the city noted 
web for its famous bridge problem (which will be. studied in Section 7,5). He became professor of mathematics 
at tiie Academy in St. Petersburg in 1725. In 1728 Goldbach went to Moscow to tutor the son of the tsar. He 
entered the world of politics when, in 1742, he became a staff member in the Russian Ministry of Foreign 
Affairs. Goldbach is best known for his correspondence with eminent mathematicians, including Euler and 
Bernoulli, for his famous conjectures in number theory, and for several contributions to analysis. 
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d) If it snows today, the university will close. The uni¬ 
versity is not closed today. Therefore, it did not 
snow today. 

e) If 1 go swimming, then I will stay in the sun too 
long. If I stay in the sun too long, then I will sun¬ 
burn. Therefore, if I go swimming, then I will sun¬ 
burn, 

2. What rule of inference is used in each of the following 
arguments? 

a) Kangaroos live in Australia and are marsupials. 
Therefore, kangaroos are marsupials. 

b) It is either hotter than 100 degrees today or the pol¬ 
lution is dangerous. It is less than 100 degrees out¬ 
side today. Therefore, the pollution is dangerous. 

c) Linda is an excellent swimmer. If Linda is an ex¬ 
cellent swimmer, then she can work as a lifeguard. 
'Therefore, Linda can work as a lifeguard. 

d) Steve will work at a computer company this sum¬ 
mer. Therefore, this summer Steve will work at a 
computer company or he will be a beach bum. 

e) If I work all night on this homework, then I can an¬ 
swer all the exercises. If! answer all the exercises, 

I will understand the material. Therefore, if I work 
all night on this homework, then I will understand 
the material. 

3. Construct an argument using rules of inference to show 
that the hypotheses “Randy works hard;' “If Randy 
works hard, then he is a dull boy,” and “If Randy is 
a dull boy, then he will not get the job" imply the con¬ 
clusion “Randy will not get the job." 

4. Construct an argument using rules of inference to show 
that the hypotheses "If it does not rain or if it is not 
foggy, then the sailing race will be held and the life- 
saving demonstration will go on," “If the sailing race 
is held, then the trophy will he awarded," and The 
trophy was not awarded" imply the conclusion “It 
rained.” 

5. What mles of inference are used in the following fa¬ 
mous argument? “AH men are mortal. Socrates is a 
man. Therefore, Socrates is mortal." 

6. What rules of inference are used in the following ar¬ 
gument? “No mart is an island. Manhattan is an island. 
Therefore, Manhattan is not a man," 

7. For each of the following sets of premises, what rele¬ 
vant conclusion or conclusions can be drawn? Explain 
the rules of inference used to obtain each conclusion 
from the premises, 

a) "If 1 take the day off, it either rains or snows.” “! 
took Tuesday off or 1 took Thursday off.” “It was 
sunny on Tuesday.” “It did not snow on Thursday.” 

b) “If 1 eat spicy foods, then I have strange dreams.” 

“1 have strange dreams if there is thunder while 1 
sleep.” "I did not have strange dreams.” 

c) “I am either clever or lucky," T am not lucky." "If 
! am lucky, then 1 will win the lottery." 


d) “Every computer science major has a personal com¬ 
puter.” “Ralph does not have a personal computer," 
“Ann has a personal computer.” 

e) “What is good for corporations is good for the 
United Slates.” “What is good for the United States 
is good for you.” "What is good for corporations is 
for you to buy lots of stuff.” 

0 “All rodents gnaw their food.” “Mice are rodents." 
“Rabbits do not gnaw their food.” “Bats are not ro¬ 
dents.*' 

8. For each of the following sets of premises, what rele¬ 
vant conclusion or conclusions can be drawn? Explain 
the rules of inference used to obtain each conclusion 
from the premises. 

a) “If 1 play hockey, then I am sore the next day." “1 
use the whirlpool if f am sore.” "I dtd not. use the 
whirlpool." 

b) “If I work, it is either sunny or partly sunny." "1 
worked last Monday or 1 worked last Friday." “It 
was not sunny on Tuesday.’’ “It was not partly sunny 
on Friday.” 

c) “All insects have six legs.” “Dragonflies are in¬ 
sects.*’ "Spiders do not have six legs." “Spiders eat 
dragonflies.” 

d) “Every student has an Internet account.” “Homer 
docs not have an Internet account.” “Maggie has an 
Internet account." 

e) "All foods that are healthy to eat do not taste good." 
■Tofu is healthy to eat." “You only eat what tastes 
good.” “You do not tat tofu.” “Cheeseburgers are 
net healthy to eat.” 

f) “I am either dreaming or hallucinating." “I am not 
dreaming.” “If I am hallucinating, I see elephants 
running down the road.” 

9. For each of the following arguments, explain which 
rules of inference are used for each step, 
at “Doug, a student in this class, knows how to w rite 
programs in JAVA. Everyone who knows how to 
write programs in JAVA can get a high-paying job. 
Therefore, someone in this class can get a high- 
paying job." 

b) “Somebody in this class enjoys whale watching, 
Every person who enjoys whale watching cares 
about ocean pollution. Therefore, there is a person 
in this class who cares about ocean pollution." 
cj “Each of the 93 students in this class owns a per¬ 
sonal computer Everyone who owns a personal 
computer can use a word processing program. 
Therefore, Zeke, a student in this class, can use a 
word processing program.” 

d) “Everyone in New Jersey lives within 50 miles 
of the ocean. Someone in New Jersey has never 
seen the ocean. Therefore, someone who lives 
within 50 miles of the ocean has never seen the 
ocean.” 
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10. For each nf the following arguments, explain which 
rules of inference are used for each step. 

a) "Linda, a student in this class, owns a red convert¬ 
ible, Everyone who owns ;l red convertible has got¬ 
ten at least one speeding ticket. Therefore, someone 
in this class has gotten a speeding ticket,” 

b) "Each of five roommates, Melissa, Aaron, Ralph, 
Veneesha, and Keeshawn, has taken a course in dis¬ 
crete mathematics. Every student who has taken a 
course in discrete mathematics can take a course in 
algorithms. Therefore, all five roommates can take 
a course in algorithms next year." 

ci “AH movies produced by John Sayfes are wonder¬ 
ful. John Sayles produced a movie about coal min¬ 
ers, Therefore, there is a wonderful movie about 
coal miners." 

d) ‘There is someone in this class who has been to 
France. Everyone who goes to France visits the 
Louvre Therefore, someone in this class has vis¬ 
ited the Louvre." 

11, Determine whether each of she following arguments re 
valid. If an argument is correct, what rale of inference 
is being used? If it is not, what fallacy occurs? 

a) If rr is a real number such that n > 1, then rr > l. 

Suppose that n 1 > 1. Then n > !. 
hi The number log: 3 is irrational if it is not the ratio 
of two integers. Therefore, since log; 3 cannot be 
written in the form alb where a and b are integers, 
it is irrational. 

c) if n is a real number with n > 3, then tr > 9. Sup¬ 
pose that ir < 9. Then n % 3. 

d) A positive integer is either a perfect square or it has 
uneven numberofpositive integer divisors. Suppose 
that n is a positive integer that has an odd number of 
positive integer divisors. Then/i is a perfect square, 

t) If n is a real number with n > 2. then rr > 4. Sup¬ 
pose that n ?s 2, Then rr £ 4. 

12. The following argument is an incorrect proof of the the¬ 
orem "(for is not divisible by 3, then n is not divisible 
by 3,” The reason it is incorrect is that circular reason¬ 
ing has been used. Where has the error in reasoning 
been made? 

[i rr is not divisible by 3, then rr does not equal 3k 
for some integer L Hence, n does not equal 3/ for 
some integer l. Therefore, n is not divisible by 3. 

13. Prove the proposition P(0), where P(n) is the propo¬ 
sition “If n is a positive integer greater than i, then 
n- > n," What kind of proof did you use? 

14. Prove the proposition Ft 1), where Pin ) is the proposi¬ 
tion “If rt is a positive integer, then it 1 ? What kind 
of proof did you use'.' 

15. Let Pint be the proposition “If a and b are positive real 
numbers, then (a +- bf -- a " + b"" Prove that P{ I) is 
true. What kind of proof did you use? 


16. Prove that the square of an even number is an even 
number using 

ai a direct, proof. 

hi an indirect proof. 

c) a proof by contradiction, 

17. Prove that if n is an integer and « 3 + 5 is odd. then ft is 
even using 

a) an indirect proof. b) a proof by contradiction 

18. Prove that if rt is an integer and 3n + 2 is even, then n 
is even using 

a) an indirect proof, b) a proof by contradiction. 

19. Prove that the sum of two odd integers is even. 

20. Prove that the sum of two rational numbers is rational. 

21. Prove that the sum of an irrational number and a ratio¬ 
nal number is irrational using a proof by contradiction, 

22. Prove that the product of two rational numbers is ratio¬ 
nal. 

23. Prove or disprove that the product of two irrational 
numbers is irrational, 

24. Prove or disprove that the product of a nonzero rational 
number and an irrational number is irrational. 

*25. Prove or disprove that rr - 79rr + 1601 is prime when¬ 
ever n is a positive integer. 

26. Prove or disprove that 2* i- 1 is prime for all nonnega¬ 
tive integers n. 

27. Show that v'3 is irrational. 

*28. Shew that Jn is irrational if n re a positive integer that 
is not a perfect square. 

29. Prove that if x and y are real numbers, then inaxl x, y) + 
•mm .v. y) = x + y. (Hint: Use a proof by cases, with 
the two cases corresponding to x a y and r < y, re¬ 
spectively.) 

30. Prove that the square of an integer not divisible by 5 
leaves a remainder of 1 or 4 when divided by 5. (Hint: Use 
a proof by cases, where the cases correspond to the pos¬ 
sible remainders for the integer when it is divided by 5.) 

31. Prove that if x and y are Teal numbers, then |Vj+ |y| a 
|.r -v y| ( where |x[ represents the absolute value of x, 
which equals a if jr >0 and equals -a if a s 0). 

32. Use a proof by cases to show that [w/2]fn/21 = [n 3 /4j 
for all integers n. 

33. Use a proof by cases to show that min(a. min(6, c)} = 
min(min(a, b), t) whenever «, A and care real numbers. 

34. Prove that if n is a positive integer, then it is even if and 
only if 7n + 4 is even. 

35. Prove that if n is a positive integer, then n is odd if and 
only if 5 /j + 6 is odd. 

36. Prove that m 2 - rt 1 if and only if m - norm --- - n. 

*37. Let p lx’ prime. Prove that a 1 ~ b 1 (mod p) if and only 

if <J = b (mod p) or a = ~b (mod p). 

38. Prove or disprove that n 2 — 1 is composite whenever/? 
is a positive integer greater than 1. 

39. Prove or disprove that if m and n are integers such that 
mn - 1, then either m = I and n = 1, or else m - 
- I and n = - I, 
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41). Prove or disprove that a mod m + h mod m = (a + 

b) mod m whenever n is a positive integer. 

41. Prove or disprove that every positive integer can be 
written as the sum of the squares of two integers, 

42. Prove that if n is a positive integer such that the sum of 
its divisors is n + I, then n is prime. What kind of proof 
did you use? 

43. Prove or disprove each of the following statements 
about the floor and ceiling functions, 

a ) Fkl = [rj for all real numbers x. 

b) |2.rj = 2[x\ whenever x is a real number, 

c) r.v] +■ r.vl - fur + vl = 0 or 1 whenever x and y are 
real numbers, 

d) Uyl - I'.ilfy] for all real numbers x and y. 

e) | k - J — ! for all real numbers r, 

44. Prove or disprove each of the following statements 
about the floor and ceiling functions. 

a) [f.rlj = [jr for all real numbers .r. 

b) |.r + yj - |ij + [yj for all real numbers r and y. 

c) f xf2]t2) - f.r/4] for all real numbers x. 

d) (c[.r]| - [ v -V j for all real numbers x. 

cl i.v| + [ v| -1 -v i yj £ [2.vj - [2yj for all real numbers 
x and v. 

45. Prove that if x is a positive real number, then 

a) L ,k'j - L-4 
bl fv'Wi = \ a' 

46. Prove that if /?i and n are positive integers and x is a 
real number, then 

| |xj I n : _ I a + n I 

l J 1 W j' 

*47. Prove that if m is a positive integer and x is a real num¬ 
ber, then 


ml m 


**48, Show that if a and b are positive irrational numbers 
such that 1/a + Ub - 1. then every positive integer 
can be uniquely expressed as either[f T aJ or [kb\ for some 
positive integer k. 

49. Prove that at least one of the real numbers a t , at ,..., a„ 
is greater than or equal to the average of these numbers. 
What kind of proof did you use? 

*50. Use Exercise 49 to show that if the first 10 positive in¬ 
tegers are placed around a circle, in any order, there 
exist three integers inconsecutive locations around the 
circle that have a sum greater than or equal to 17. 

51. Prove that if n is an integer, the following four state¬ 
ments are equivalent: (i) n k even, (ii) n + 1 is odd, 
(iii) 3u + I is odd, (iv) 3« is even. 


52. Prove that if n is an integer, the following three state¬ 
ments are equivalent: (1)5 divides n, (ii) 5 divides n 1 . 
(iii) n 2 ^ ± 1 (mod 5), 

53. Prove or disprove that there are three consecutive odd 
positive integers that are primes, that is, odd primes of 
the form p, p + 2, and p + 4. 

54. Prove or disprove that given a positive integer n, there 
are n consecutive odd positive integers that are primes 

55. Which rules of inference are used to establish the con¬ 
clusion of Lewis Carroll’s argument described in Ex¬ 
ample 20 of Section 1.3? 

56. Which rules of inference are used to establish the con¬ 
clusion of Lewis Carroll’s argument described in Ex¬ 
ample 21 of Section 1.3? 

57. Give a constructive proof of the proposition: “For every 
positive integer n there is an integer divisible by more 
than n primes,” 

58. Find a counterexample to the proposition: "For every 
prime number n, n (-2 is prime.” 

*59. Prove that there are infinitely many primes congruent 
to 3 modulo 4. Is your proof constructive or noncon¬ 
structive? (Him: One approach is to assume that there 

are only finitely many such primes p t , p+, _ p„. Let 

q = 4pi p-> ••• p„ + 3. Show' that q must have a prime 
factor congruent to 3 modulo 4 not among the n primes 
Pi, Pi . Pn) 

68. Prove or disprove that if p\, pi, - p„ are the n small¬ 
est pnmes, then p\pi'" }h +■ 1 is prime, 

61. Show that (he propositions p u pi, p\, pi, and ps can be 
shown to be equivalent by pro ving that the implications 
Pi -* Pi. ps -* pi, pt -* p 2 , P2 Pi, and p 5 

pi are true, 

62. Prove or disprove that if a and b are rational numbers, 
dien (F is also rational. 

63. Prove that there are irrational numbers a and b such 
that a h is rational. Is your proof constructive nr non¬ 
constructive? [Him Let a = J2 and b - J2. Show 
that either a' 1 or (u b 'f is rational.J 

64. Prove that an 8 X 8 chessboard can be completely cov¬ 
ered using dominos (1x2 pieces). 

*65. Prove that it is impossible to cover completely with 
dominos the 8x8 chessboard with two squares at op¬ 
posite comers of the board removed. 

*86. The Logic Problem, taken from WFF’N PROOF, The 
Game of Logic, has the following two assumptions: 

1. “Logic is difficult or not many students like 
logic.” 

2, “If mathematics is easy, then logic is not difficult,” 
By translating these assumptions into statements in¬ 
volving propositional variables and logical connec¬ 
tives, determine whether each of the fallowing are valid 
conclusions of these assumptions: 

a) That mathematics is not easy, if many students like 
logic. 


PDF created with pdfFactory trial version www.pdffactorv.com 




186 31 Mathematical Reasoning 


b) That not many students like logic, if mathematics 
is not easy, 

c) That mathematics is not easy or logic is difficult. 

d) That logic is not difficult or mathematics is not 
easy. 

e) That if not many students like logic, then either 
mathematics is not easy or logic is not difficult. 

*67, Determine whether the following argument, taken from 

Backhouse [Ba86], is valid. 

If Superman were able and willing to prevent 
evil, he would do so. If Superman were unable 
to prevent evil, he would be impotent: if he were 
unwilling to prevent evil, he would be malevo¬ 
lent. 


Superman does not prevent evil. ]f Superman ex¬ 
ists, he is neither impotent nor malevolent. There¬ 
fore, Superman does not exist. 

Resolution is a proof method used extensively in artificial 
intelligence and by automated proof programs, Resolution 
is based on the rule of inference derived front the tautology 
((p v q) A (~ip V 'vi * (4 V')• 

68, Use the resolution rule of inference to prove the state¬ 
ment “You will win the lottery or you wall be pro¬ 
moted,” given the hypotheses “You will quit your job 
or you will win the lottery,” “You will not quit your job 
or you will find a better job,” and “You will not find a 
better job or you will be promoted." 


12 _ 

Mathematical Induction 


INTRODUCTION 


What is a formula for the sum of the first n positive odd integers? The sums of the first 
n positive odd integers for n = 1,2,3, 4,5 are 

1-1, 1+3-4, 1 +3 + 5 = 9, 

1+3 + 5 + 7=16, I + 3 + 5 + 7 + 9 = 25, 

From these values it is reasonable to guess that the sum of the first n positive odd integers 
is ft 2 . We need a method to prove that this guess- is correct, if in fact it is. 

Mathematical induction is art extremely important proof technique that can be 
used to prove assertions of this type. As we will see in this section and in subsequent 
chapters, mathematical induction is used extensively to prove results about a large va¬ 
riety of discrete objects. For example, it is used to prove results about the complexity 
of algorithms, the correctness of certain types of computer programs, theorems about 
graphs and trees, as well as a wide range of identities and inequalities. 

In this section we will describe how- mathematical induction can be used and why 
it is a valid proof technique. It is extremely important to note that mathematical induc¬ 
tion can be used only to prove resuits obtained in some other way. It is not a tool fur 
discovering formulae or theorems. 


THE WELL-ORDERING PROPERTY 


The validity of mathematical induction follows from the following fundamental axiom 
about the set of integers. 

The Well-Ordering Properly Every nonempty set of nonnegative integers has a 
least element 


The well-ordering property can often be used directly in proofs. 
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Use the well-ordering property to prove the division algorithm. Recall that the division 
algorithm states that if a is an integer and d is a positive integer, then there are unique 
integers q and r with 0 ■< r <d and a — dq + r. 

Solution: Let S be the set of nonnegative integers of the form a - dq where q is an 
integer. This set is nonempty since —dq can be made as large as desired (taking q to 
be a negative integer with large absolute value). By the well- ordering property S has a 
least element r - a- dq$. 

The integer r is nonnegative. It is also the case that r < d. If it were not, then there 
would be a smaller nonnegati ve element in S. namely, a-d(qn + 1). To see this, suppose 
that r == d. Since,-? - Jq l} + 1, it follows that a - d(q$ ^ 1 ) - (a-dq a )-d - r~d - 
0. Consequently, there are integers q and r with 0 ^ r < d. The proof that q and r are 
unique is left as an exercise for the reader. ■ 


MATHEMATICAL INDUCTION 

Many theorems state that Pin) is true for all positive integers it, where Pi n) is a propo¬ 
sitional function, such as the statement that 1 4 2+ ■ ■ ■ + n = n(n+ 1 )/2 or the statement 
that n <= 2". Mathematical induction is a technique for proving theorems of this kind. In 
other words, mathematical induction is used to prove propositions of the form Pin), 
where the universe of discourse is the set of positive integers. 

A proof by mathematical induction that P(n) is true for every positive integer n 
consists of two steps; 

/. Basis step. The proposition Pi 1) is shown to be true. 

2. Inductive step. The implication Pin) —» P(n + 1) is shown to be true for every 
positive integer n. 

Here, the statement Pin) for a fixed positive integer n is called the inductive hypothe¬ 
sis. When we complete both steps of a proof by mathematical induction, we have proved 
that Pin) is true for all positive integers n; that is, we have shown that Vn Pin) is true. 
Expressed as a rule of inference, this proof technique can be stated as 

[P(l)AVn(7>(n) - Pin 4 1))] • Vti Pin). 

Since mathematical induction is such an important technique, it is worthwhile to explain 
in detail the steps of a proof using this technique. The first thing wc do to prove that P(n) 
is true for all positive integers n is to show that P{]) is true. This amounts to showing 
that the particular statement obtained when n is replaced by 1 in Pin) is true. Then we 
must show that Pi n) —■ Pin 4 l) is true lor every positive integer n. To prove that this 
implication is true for every positive integer n, we need to show that Pin + 1) cannot 
be false when P(n) is true. This can be accomplished by assuming that P(n) is true and 
showing that under this hypothesis P(n 4 1) must also be true. 

Remark: In a proof by mathematical induction it is not assumed that Pi n) is true for all 
positive integers! It is only shown that if it is assumed that Pin) is true, then P{n + 1) is 
also true. Thus, a proof by mathematical induction is not a case of begging the question, 
or circular reasoning. 

When we use mathematical induction to prove a theorem, we first show that Pi 1) is 
true. Then we know' that Pi2) is true, since Pi 1) implies P(2). Further, we know that 
F(3) is true, since P( 2) implies P( 3). Continuing along these lines, we see that Pi k) is 
true, for any positive integer k. 
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There are several useful illustrations of mathematical induction that can help you 
remember how this principle works. One of these involves a line of people, person one, 
person two, and so on. A secret is told to person one, and each person tells the secret 
to the next person in line, if the former person hears it. Let Pin) be the proposition that 
person n knows the secret. Then P( 3) is true, since the secret is told to person one; P( 2) 
is true, since person one tells person two the secret; P(3>) is true, since person two tells 
person three the secret; and so on. By the principle of mathematical induction, every 
person in line learns the secret. This is illustrated in Figure 1. (Of course, it has been 
assumed that each person relays the secret in an unchanged manner to the next person, 
which is usually not true in real life.) 

Another way to illustrate the principle of mathematical induction is to consider an 

infinite row of dominos, labeled J, 2,3_, n. where each domino is standing up. Let 

Pi n) he the proposition that domino n is knocked over. If the first domino is knocked 
over—i.e,, if Pi 1) is true—and if, whenever the nth domino is knocked over, it also 
knocks the in + 1 )th domino over—i.e., if Pin) -» Pin + 1) is true—then all the domi¬ 
nos are knocked over. This is illustrated in Figure 2. 

Why Mathematical Induction Is Valid Why is mathematical induction a valid 
proof technique? The reason comes from the well-ordering property. Suppose we know 
that P(l) is true and that the proposition Pin) -» Pin + I) is true for all positive 
integers n. To show that Pin) must be true for all positive, integers, assume that there 
is at least one positive integer for which P\n) is false. Then the set S of positive 
integers for which P(n) is false is nonempty. Thus, by the well-ordering property, S has 
a least element, which will be denoted by k. We know 1 that k cannot be 1, since Pi 1) 
is Ime. Since k is positive and greater than !, k - 1 is a positive integer. Furthermore, 
since k - 1 is less than k, it is not in S. so P(k - i) must be true. Since the implication 
P( k - 1) —* P(k) is also true, it must be the case that P(k) is true. This contradicts the 
choice of k. Hence, Pi n) must be true for every positive integer n. 


Historical note: The first known use of mathematical induction is in the work of the sixteenth-century 
mathematician Francesco Maurolico (1494-1575). Maurolico wrote extensively on the works of classical 
mathematics and made many contributions to geometry and optics. In his book Anlhmelicomm LibriDuo 
Maurolico presented a variety of properties of the integers together with proofs of these properties. To prove 
some of these properties he devised the method of' mathematical induction, His first use of mathematical 
induction in this book was to prove that the sum of the first n odd positive integers equals ft 2 . 
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FIGURE 2 Illustrating How Mathematical Induction Works Using Dominos. 


EXAMPLES OF PROOFS BY MATHEMATICAL INDUCTION 


We will use a variety of examples to illustrate how theorems are proved using mathe- 
wib matieal induction. We begin by proving a formula for the sum of the first n odd positive 
• integers. (Many theorems proved in this section via mathematical induction can be 
proved using different methods. However, it is worthwhile to try to prove a theorem in 
more than one way, since one method of attack may succeed whereas another approach 
may not.) 


EXAMPLE 2 Use mathematical induction to prove that the sum of the first n odd positive integers 

is n 2 . 


Solution: Let P{n) denote the proposition that the sum of the first n odd positive integers 
is n 1 . We must first complete the basis step; that is, we must show' that P( 1) is true. Then 
we must carry out the inductive step; that is, we must show that f'(n + 1) is true when 
Pin) is assumed to be true. 

PAS IS STEP: P(V) states that the sum of the first one odd positive integers is \ 2 . 
This is true since the sum of the first odd positive integer is 1. 

INDUCTIVE STEP: To complete the inductive step we must show that the propo¬ 
sition Pin) —<■ Pin + 1) is true for every positive integer n. To do this, suppose that 
Pin) is true for a positive integers; that is, 

! + 3 + 5 + ■ ■ ■ + (2n - 1) = tr. 

[Note that the nth odd positive integer is (2n - 1), since this integer is obtained by 
adding 2 a total of n - 1 times to L] We must show that Pi n + 1) is true, assuming that 
Pin) is true. Note that Pin 4 1) is the statement that 

1 + 3 + 5 + ■ ■ - 4 (2n - 1) + (2n 4 1) = (n + l) 2 . 
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EXAMPLE 3 


EXAMPLE 4 


So, assuming that P(n) is true, it follows that 

1 + 3 + 5 + ■■■+■ (2n - 1) + (2a + I) - [1 + 3 +■■■+- (2n - 1)] + (2 n + 1) 

= n 2 4- (2n + 1) 

= fl? -h 2fl + 1 

= (« +l) 2 . 

This shows that Pin + 1) follows from Pin). Note that we used the inductive hypothesis 
Pi m in the second equality to replace the sum of the first n odd positive integers by n 1 . 

Since Pi 1} is true and the implication Pit)) P(n + 1) is true for all positive in¬ 
tegers n, the principle of mathematical induction shows that Pin) is true for all positive 
integers n ■ 


The next example uses the principle of mathematical induction to prove an 
inequality. 


Use mathematical induction to prove the inequality 
« < 2 " 

for all positive integers n. 

Solution: Let Pin) be the proposition “n < 2" " 

BASIS STEP; P{ 1) is true, since 1 < 2 1 = 2. 

INDUCTIVE STEP: Assume that Pin) is true for the positive integer n That is, 
assume that n < 2". We need to show' that Pin + 1) is true. That is, we need to show' 
that n + 1 < 2* +! . Adding ] to both sides of n < 2", and then noting that 1 s 2 n , 
gives 

n f 1 < 2" + 1 < 2" v 2" = 2 n+l . 

We have shown that P(n +1) is true, namely, that n 4 3 < 2' !+1 , based on the assumption 
that Pin) is true, The induction step is complete. 

Therefore, by the principle of mathematical induction, it has been shown that n < 
2” is true for ail positive integers n. ■ 


We will now use mathematical induction to prove a theorem involving the divisi¬ 
bility of integers. 


Use mathematical induction to prove that n 3 - n is divisible by 3 whenever n is a positive 
integer. 

Solution: To construct the proof, let Pin) denote the proposition: “n 3 - n is divisible 
by 3.” 

BASIS STEP: P( l) is true, since l 3 - 1 = 0 is divisible by 3. 
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EXAMPLE 5 


INDUCTIVE STEP: Assume that P(n) is true; that is, n 3 - n is divisible by 3. We 
must show that Pin +1} is true. That is, we must show' that (n + 1 ) 3 - (n + 1) is divisible 
by 3. Note that 

(n + l) 3 - (n + 1) = (n 3 + Sri 2 + 3 n+ 1} - (ni+ I) 

= (n 3 - n) + 3 (n 2 + n). 

Since both terms in this sum are divisible by 3 (the first by the assumption of the 
inductive step, and the second because it is 3 times an integer), it follows that in + 
l) 3 - (n + l) is also divisible by 3. This completes the induction step. Thus, by the 
principle of mathematical induction, n 3 ~ n is divisible by 3 whenever n is a positive 
integer. ■ 


Sometimes we need to show' that Pin) is true for n = k, k + \, k + 2 __ where 

k is an integer other than 1. We can use mathematical induction to accomplish this as 
long as we change the basts step. For instance, consider Example 5, which proves that 
a summation formula is valid for all nonnegative integers, so that we need to prove that 
P(n) is true for n - 0,1,2,.... 


Use mathematical induction to show' that 
1 +2 + 2 2 + ”- + 2 " = 2' )+1 -1 
for all nonnegative integers n. 

Solution: Let P(n) be the proposition that this formula is correct for the integer n, 
BASIS STEP: P{ 0) is true since 2° = 1 = 2 1 - 1. 

INDUCTIVE STEP: Assume that Pin) is true. To carry out the inductive step using 
this assumption, it must be shown that Pin + 1) is true, namely, 

1 + 2 + 2 2 + ■■■ + 2 '’ + 2 n+l = 2 ( " tl,+1 - 1 = 2 ! '~ 2 - 1 , 

Using the inductive hypothesis Pin), it follows that 

1 + 2 + 2 2 + -■■ + 2 " + 2" +1 = (1 + 2 + 2 2 + ■■■ + 2 ") + 2" +1 

= ( 2 nM - 1 ) + 2' 1 ' 1 

= 2 - 2"' 1 - 1 

= 2 n+2 - 1 . 

This finishes the inductive step, which completes the proof. ■ 


As Example 5 demonstrates, to use mathematical induction to show that P{n) is 
true for n = k, k + 1, k + 2,..., where k is an integer other than 1, we show that Pik) 
is true (the basis step) and then show that the implication Pin) — Pitt + I) is true for 
n = k, k+ 1, k+ 2,... (the inductive step). Note that k can be negative, zero, or positive. 
Following the domino analogy we used earlier, imagine that we begin by knocking 
down the Ath domino (the basis step), and as each domino falls, it knocks down the 
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EXAMPLE 6 


next domino (the inductive step). We leave it to the reader to show that this form of 
induction is valid (see Exercise 6SS). 

The formula given in Example 5 is a special case of a general result for the 
sum of the terms of a geometric progression, which is a sequence of the form 

a, ar, ar 1 . ar n . where a and r axe real numbers. For instance, the sequence 

in Example 5 is a geometric progression with a - 1 and r = 2. Likewise, the se¬ 
quence 3, 15, 75.3 ■ 5",... is a geometric progression with a - 3 and r - 5. The 

next example gives a formula for the sunt of the first n + 1 terms of such a sequence. 
The proof of this general formula will use mathematical induction. 


Sums of Geometric Progressions Use mathematical induction to prove the following 
formula for the sum of a finite number of terms of a geometric progression: 

V' , i „ aP r] - a 

> ar 1 = a * ar + ar + • ■ ■ + ar = -, when t t 6 1. 

U r - 

Solution: To prove this formula using mathematical induction, let P(n) be the propo¬ 
sition that the sum of the first n + 1 terms of a geometric progression in this formula is 
correct. 

BASIS STEP: P(Q) is true, since 

ar - a 

a =-- . 

r - 1 

INDUCTIVE STEP: Assume that Pin) is true. That is, assume 

i „ ar n ~ l - a 

a + ar + ar + ■ * ■ + nr =-. 

r - 1 

To show that this implies that Pin + 1) is true, add ar'* 1 to both sides of this equation 
to obtain 


1 , ,,i ar - a „ , 

a + ar + ar t ■ ■ ■ + ar ~ ar = -+ ar , 

r - 3 

Rewriting the right-hand side of this equation shows that 

ar "" 1 - a +1 ar " 41 - a ar n+2 - «r " +1 
-+ ar =-+- 

r ~ 1 r - 1 r - 1 

_ ar n+2 - a 
r - 1 ’ 

Combining these equations gives 

i , . „_i ur n *- - a 

a + or f ar + ■ ■ ■ + ar + ar = - - -. 

r - 1 

This shows that if P(n) is true, then Pin + 1) must also be true. This completes the 
inductive argument and shows that the formula for the sum of the terms of a geometric 
scries is correct. ■ 
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EXAMPLE 7 


As previously mentioned, the formula in Example 5 is the case of the formula in 
Example 6 with a = 1 and r = 2. The reader should verify that putting these values 
for a and r in the general formula gives the same formula as in Example 5, 

An important inequality for the sum of the reciprocals of a set of positive integers 
will be proved in the next example. 


An Inequality for Harmonic Numbers The harmonic numbers k = 1,2,3. 
are defined by 


Hi = 1 


1 1 1 

2 + 3 + + *' 


For instance. 




25 

12 ' 


Use mathematical induction to show that 


H 2 * a 1 + 


ft 

2 ' 


whenever n is a nonnegative integer. 


Solution: To carry out the proof, let Pin) be the proposition that H 2 * ^ I + nil. 

BASIS STEP: P(0) is true, since /7w = H\ = I 2: 1 + 0/2. 

INDUCTIVE STEP: Assume that Pin) is true, so that - I + nil. It must be 
shown that Pin + 1), which, states that // r , r I (n + l)/2, must also be true under 
this assumption. This can be done since 


H 



Or. 



1 


= H 


+ 


] 

2 « t I 


+ ■■■ + 


1 

2TTT 



1 

pTT 


(by the inductive hypothesis) 



(since there are 2 ,: terms each not less than J/2 ,: 1 ) 


,n + |H 


ft + 1 

~T ‘ 


This establishes the inductive step of the proof. Thus, the inequality lor the harmonic 
numbers is valid for all nunnegative integers n. ■ 
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EXAMPLE 8 


EXAMPLE 9 


Remark: The inequality established here ean be used to show that the harmonic series 


1 + 


1 

2 


+ 


1 

3 


+ 


1 

+ — +■ * 1 ' 

n 


is a divergent infinite series. This is an important example in the study of infinite series. 


The next example shows how mathematical induction can be used to verify a for¬ 
mula for the number of subsets of a finite set. 


The Number of Subsets of a Finite Set Use mathematical induction to show' that if S 
is a finite set with n elements, then S has 2 n subsets. (We will prove this result directly 
in several ways in Chapter 4.) 

Solution: Let Pin) be the proposition that a set with n elements has 2 s subsets. 

BASIS STEP: 7(0) is true, since a set with zero elements, the empty set, has exactly 
2° - 1 subsets, since it has one subset, namely, itself. 

INDUCTIVE STEP: Assume that Pin) is true, that is, that every set with n elements 
has 2 n subsets. It must be shown that under this assumption Pin + 1), which is the 
statement that every 1 set with n + 1 dements has 2' t+1 subsets, must also be true. To 
show this, let T be a set with n + 1 elements. Then, it is possible to write T — S U {a} 
where a is one of the elements of T and S - T- {a}. The subsets of T can be obtained 
in the following way. For each subset X of S there arc exactly two subsets of T, namely, 
X and X U {a}. (This is illustrated in Figure 3.) These constitute all the subsets of 7 and 
are all distinct. Since there are 2" subsets of S, there are 2-2" = 2" +1 subsets of 7, 
This finishes the induction argument. ■ 


Show that if n is a positive integer, 

1 + 2 + ■ -- + n = n(n + l)/2. 


FIGURE 3 Generating Subsets of a Set with n + 1 Elements. Here T = 8 U {«}. 
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Solution: Let Pin) be the proposition that the sum of the first n positive integers is 

n(n +1 )/2. We must do two things to prove that Pin) is true forn = 1,2, 3.Namely, 

we must show that Pi 1) is trae and that the implication Fin) implies Pin + 1) is true 
for n = 1,2,3. 

BASIS STEP: P( l ) is true, since 1 = 1(1 + l)/2, 

INDUCTIVE STEP: Assume that Pin) holds so that 

1 + 2 + ■ ■ ■ ■+ n = n(n + l)/2. 

Under this assumption, it must be shown that Pin 4- 1} is trae, namely, that 
l + 2 + ■■■ 4- n + n + 1 = (n + l)[(n + 1) + 1 ]/2 = (n + l)(n + 2)/2 
is also true. Add n + 1 to both sides of the equation in Pin) to obtain 

1 + 2 + ■ ■- + n + (n + 1) = + 1)12 + (n + 1) 

= [(nil) + l](n 4- 1) 

= in + [){n 4 21/2. 

This last equation shows that Pin + 1) is trae. This completes the inductive step and 
completes the proof. g 

EXAMPLE tO 

Use mathematical induction to prove that 2" < «! for every' positive integer n with 
n s 4. 

Solution: Iel Pin) be the proposition that 2 n < «!. 

BASIS STEP: To prove the inequality for n s 4 requires that the basis step be F(4). 
Note that P(4) is true, since 2 4 = 16 < 4! = 24. 

INDUCTIVE STEP: Assume that Pin) is trae. That is, assume that 2" < «!. We 
must show that PI a 4 1}is tme. That is, we must show that 2 re +1 <(«+!)!. Multiplying 
both sides of the inequality 2™ < ri! by 2, it follows that 

2 ■2" < 2 n\ 

< in 4 1) ■ n! 

= (nr I)!. 

This shows that P(n 4 1) is true when Pin) is true. This completes the inductive step 
of the proof. Hence, it follows that 2 n < n! is true for all integers n with n s 4. ■ 

EXAMPLE 11 

Use mathematical induction to prove the following generalization of one of De Mor¬ 
gan's laws: 

i>*- 0^ 

Jfc= i * - 1 

whenever A h Ai — ,A„ are subsets nf a universal set U and n s 2. 

Solution: Let Pin ) be the identity for n sets. 
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BASIS STEP: The statement P(2) asserts that A\ D A 2 - A ] U A 2 . this is one of 
De Morgan’s laws; it was proved in Section 1,5. 

INDUCTIVE STEP: Assume that Pin) is true, that is, 


Pr 




It 


U'C 


whenever At, Ai,., A„ are subsets of (he universal set U. To cany out the inductive 
step it must be shown that if this equality holds for any n subsets of U, it must also be 

valid for any n 4 I subsets of U. Suppose that .4[, A 2 . A„, A, H j tire subsets of U. 

When the inductive hypothesis is assumed to hold, it follows that 




f] A*jn a„, i 
HaJua” 1 


(by De Morgan's lawl 


- [J-M JU Ajf-i 


(by the inductive hypothesis) 


= U : ^ 

k= l 

This completes the proof by induction. ■ 


The next example illustrates how mathematical induction can be used to prove a 
W$b result about covering chessboards with pieces shaped like the letter "L." 

EXAMPLE 12 Let n be a positive integer. Show that arty 2" X 2 r ' chessboard with one square removed 

can be tiled using L-shaped pieces, where these pieces cover three squares at a time, as 
shown in Figure 4. 

Solution: Let Pin) be (he proposition that any 2” X 2 n chessboard with one square 
removed can be tiled using L-shaped pieces. We can use mathematical induction to 
prove that Pin) is true for all positive integers n. 



FIGURE 4 An L-Shaped Piece. 
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FIGURE 5 Tiling 2 >■ 2 Chessboards with One Square Removed. 


BASIS STEP: P{[) is true, since any of the four 2 X 2 chessboards vviih one square 
removed can be tiled using one L-shaped piece, as shown in Figure 5, 

INDUCTIVE STEP: Assume that Pin) is irue: that, is, assume that any 2" X 2' 1 
chessboard with one square removed can be riled using L-shaped pieces. It must be 
shown that under this assumption P(t i + 1} must also be true; that is, any 2"' 1 x 2"' : 
chessboard with one square removed can be tiled using L-shaped pieces. 

To see this, consider a 2* 11 x2'" chessboard with one square removed. Split 
this chessboard into four chessboards of size 2" X 2", by dividing it in half in both 
directions. This is illustrated in Figure 6. No square has been removed from three of 
these four chessboards. The fourth 2" x2 ri chessboard has one square removed, so by the 
inductive hypothesis, it can be covered by L-shaped pieces. Now temporarily remove 
the square from each of the other three 2" x 2 n chessboards that has the center of the 
original, larger chessboard as one of its comers, as shown in Figure 7. By the inductive 
hypothesis, each of these three 2" X 2” chessboards with a square removed can be tiled 
by L-shaped pieces. Furthermore, the three squares that were temporarily removed can 
be covered by cme L-shaped piece. Hence, the entire 2 H+1 X 2 n ~ l ' ] chessboard can be 
tiled with L-shaped pieces. This completes the proof. ■ 


THE .SECOND PRINCIPLE OF MATHEMATICAL INDUCTION 


There is another form of mathematical induction that is often useful in proofs. With this 
form we use the same basis step as before, but we use a different inductive step. We 
assume that P(k) is true for lc - 1, — n and show that Pi n + I) must also be true based 
on this assumption. This is called the second principle of mathematical induction. 



FIGURE 6 Dividing a V" 1 x 2 B ~* Chess- FIGURE 7 Tiling the 2” +l x 2" 1 Chess¬ 
board into Four 2" x 2" Chessboards. board with One Square Removed. 
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EXAMPLE 13 


EXAMPLE 14 


We summarize the two steps used to show that Pin) is true for all positive inte¬ 
gers n : 

2. Basis step. The proposition P{\ ) is shown to he true. 

2. inductive step. It is shown that [P(l> A P(2) A ■ ■ - A Pi «)J —> Pin + 1) is true for 
every positive integer n. 

The two forms of mathematical induction are equivalent; that is, each can be shown 
to be a valid proof technique assuming the other. We leave it as an exercise for the 
reader to show this. We now give an example that shows how the second principle of 
mathematical induction is used. 


Show that if n is an integer greater than 1, then n can be written as the product of primes. 

Solution: Let Pin) be the proposition that n can be written as the product of primes. 

BASIS STEP: P{2) is true, si nee 2 can be written as the product of one prime, itself 
[Note that P( 2) is the first case we need to establish.] 

INDUCTIVE STEP: Assume that P( *:} is true for all positive integers k with k 
n. To complete the inductive step, it must be shown that Pin + 1) is true under this 
assumption. 

There are two cases to consider, namely, when n + 1 is prime and when n + 1 is 
composite, Jf n + 1 is prime, we immediately see that Pi n + 1) is true. Otherwise, n - 1 
is composite and can be written as the product of two positive integers a and b with 
2 s a £ b < n + 1. By the induction hypothesis, both a and b can be- written as the 
product of primes. Thus, if n +1 is composite, it can be written as the product of primes, 
namely, those primes in the factorization of a and those in the factorization of b. ■ 


Remark: Since I is a product of primes, namely, the empty product of no primes, we 
could have started the proof in Example 13 with /*( !) as the basis step. We chose not 
to do this because many people find this confusing. 

Note that Example 13 completes the proof of the Fundamental Theorem of Arith¬ 
metic. which asserts that every nonnegative integer can be written uniquely as the prod¬ 
uct of primes in nondecreasing order. We showed in Section 2.5 that an integer has at 
most one such factorization into primes. Example 13 shows there is at least one such 
factorization. 

Using the principle of mathematical induction, instead of the second principle of 
mathematical induction, to prove the result in Example 13 is difficult. However, as 
Example 14 shows, some results can be readily proved using either the principle of 
mathematical induction or the second principle of mathematical induction. 


Prove that every amount of postage of 12 cents or more can be formed using just 4-ccnt 
and 5-cent stamps. 

Solution: We will prove this result using the principle of mathematical induction. Then 
we will present a proof using the second principle of mathematical induction. Let Pin) 
be the statement that postage of n cents can be formed using 4-cent and 5-cem stamps. 
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We begin by using the principle of mathematical induction. 

BASIS STEF: Postage of 12 cents can be formed using three 4-cent stamps. 

INDUCTIVE STEP: Assume that P{ n) is true, so that postage of n cents can be 
formed using 4-cent and 5-eent stamps. If at least one 4-cent stamp was used, replace 
it with a 5-cent stamp to form postage of n + t cents, if no 4-cent stamps were used, 
postage of n cents was formed using just 5-cent stamps. Since n ' 12, at least three 
5-cent stamps were used. So, replace three 5-cent stamps with four 4-cent stamps to 
form postage of n + 1 cents. This completes the inductive step, as well as the proof by 
the principle of mathematical induction, 

Next, we will use the second principle of mathematical induction. We will show- 
that postage of 12,13,14, and 15 cents can be formed and then show how to get postage 
of n + 1 cents for n '• 15 from postage of n - 3 cents. 

BASIS STEP: We can form postage of 12, 13, 14, and 15 cents using three 4-cent 
stamps, two 4-cent stamps and one 5-cent stamp, one 4-cent stamp and two 5-cent 
stamps, and three 5-cent stamps, respectively. 

INDUCTIVE STEP: Let n £ 15. Assume that wc can form postage of k cents, 
where 12 s k s n. To form postage of n f 1 cents, use the stamps that form postage 
of n - 3 cents together with a 4-cent stamp. This completes the inductive step, as well 
as the proof by the second principle of mathematical induction. 

(There are other ways to approach this problem besides those described here. Can 
you find a solution that does not use mathematical induction?) ■ 


Remark: Example 14 shows how we can adapt the second principle of mathematical 
induction to handle cases where the inductive step is valid only for sufficiently large 
values of n. In particular, to show that P(n) is true for n = k,k+ 1, k + 2,.. where 
k is an integer, we first show that P(k), P(k + I), P(k + 2),..., P(i) are true (the basis 
step), and then we show that [P(k) A P(k + 1) A P(k + 2) A ■ - ■ A P{n)\ -* Pin + 1) 
is true for every 1 2 3 integer n £ / (the inductive step). For example, the basis step of the 
second proof in the solution of Example 14 shows that P{\2), P(13), F(!4), and F(15) 
are true. We need to prove these cases separately since the inductive step, which shows 
that [F( 12) A P( 13) A ■ ■ ■ A P(n)] —» Pin + 1), only holds when n > 15. 

We will discuss two important applications of mathematical induction in the fol¬ 
lowing sections. The first involves the definition of sequences without giving explicit 
formulae for their terms. The second involves proving that computer programs are 
correct. 


Exercises 


1. Find a formula for the sum of tire first n even positive 
integers. 

2. Use mathematical induction to prove the formula that 
you found in Exercise 1. 

3. Use mathematical induction to prove that 3 + 3 ■ 5 + 
3 ■ 5* + ■ ■ ■ + 3 ■ 5" = 3(5"’ rl - ])/4 whenever n is a 
non negative inieger. 


4. Use mathematical induction to prove that 2 2 • 7 \ 2 - 

T- -+ 2(-7)" = (1 - (-?5 n+1 )/4 whenever n is a 

nonnegative integer. 

5, Find a formula for 
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by examining the values of this expression for small 
values of «. Use mathematical induction to prove your 
result, 

6. Find a formula for 

1 I I 

1 ■ 2 1 2 -3 n{ n r 1) 

by examining the values of this expression for small 
values of n, Use mathematical induction to prove your 
result, 

7. Show that l' t2 ; - - n{n + l)(2n + l)/6 

whenever n is a positive integer. 

8. Show that I- + 2 5 * ■ ■ ■ + n J = [rt(t? + lJ/2] 1 whenever 
n is a positive integer. 

9. Prove that l 2 -t-3 2 r 5 2 -i— ,-U.n-c I ) 2 = (ft t-1 hIu- 
13(2ri + 3)/3 whenever n is a nonnegative integer,' 

10. Prove that 1 ■ 1 1 t 2 ■ 2! + • ■ ■ - n ■ n! = (n + 1)! - 1 
whenever n is a positive integer. 

*11. Show by mathematical induction that if h > - I, then 
1 -t nh ?s (1 f hT for all nonnegative integers rt. This 
is called Bernoulli’s inequality. 

12. Prove that 3" < n! whenever ti is a positive integer 
gi cuter than 6. 

13. Show that 2' 1 > tr whenever n is an integer greater 
than 4. 

14. 1 Jsc mathematical induction to prove that ft! < n" 
whenever rt is a positive integer greater than I. 

15. Prove using mathematical induction that 

l '2 + 2 " 3 !-’■■ + nift t 1 ) 

= h(» - 1 Km + 2)/3 

whenever n ts a positive integer. 

16- Use mathematical induction to prove that 

I ■ 2 ' 3 H 2 - 3 . 4 +...-!- n(fl + ij(n + 2) 

- n(n + I)(ff + 2)(fi - 3)/4. 

17. Show that I 2 - 2- i- 3- - ■■■ -r {-!)"" Ur 1 

= (- 11’ 1 J fi(n + 1 i/2 whenever ft is a positive integer. 

18. Prove that 

, 11 I , I 

-f - r a- ■ ■ ■ i- - - < 2 — 

4 9 tt 2 ft 

whenever ft is a positive integer greater than 1, 

19. Show that arty postage that is a positive integer number 
(if cents greater than 7 cents can be formed using just 
3-cert stamps and S-cent stamps. 

20, Use mathematical induction to show that 3 divides 
ft-’ + 2n whenever it is a nonnegative integer. 

21, Use mathematical induction to show that 5 divides 
fr 1 - n whenever n is a non negative integer. 

22, Use mathematical induction to show- that 6 divides 
fr 1 - it whenever ft is a nonnegative integer. 

*23. Use mathematical induction to show that n 2 - 1 is 
divisible by 8 whenever n is an odd positive integer. 


24. Use mathematical induction to show 1 that n- -7w + 12 
is nonnegative whenever n is an integer greater than 3. 

25, Use mathematical induction to prove that a set with u 
elements has n{n - 1V2 subsets containing exactly two 
elements whenever n is an integer greater than or equal 
to 2. 

*26. Use mathematical induction to prove that a set with ft 
elements has n(n - 1 )(n - 2)/6 subsets containing ex¬ 
actly three elements whenever n is an integer greater 
than or equal to 3. 

27. Use mathematical induction to prove that , j A = 
n(n f F)(2fi + l)(3rr - 3ft - 1)/3D whenever ft is a pos¬ 
itive integer, 

28. For which nonnegative integers fi is if ft! ? Prove 
your answer using mathematical induction. 

29. For which nonnegative integers n is 2ti + 3 tS 2*7 
Prove your answer using mathematical induction. 

30. Use mathematical induction to show that 1/(2n) s [1- 

3 ■ 5 ■ - - • 1 (2fi - l)]/(2 -4.2n) whenever n is a 

positive integer. 

31. a) Determine which amounts of postage can be formed 

using just 5-cent and 6-cent stamps. 

b) Prove your answer to (a) using the principle of 
mathematical induction. 

c) Prove your answer to (a) using the second principle 
of mathematical induction. 

32. Which amounts of money can be formed using just 
dimes and quarters? Prove your answer using a form 
of mathematical induction. 

33. An automatic teller machine has only $20 hills and $51) 
bills. Which amounts of money can the machine dis¬ 
pense, assuming the machine has a limitless supply of 
these two denominations of bills? Prove your answer 
using a form of mathematical induction. 

34. Use mathematical induction to prove that V" £ 2 * - 

(fi - I}!"- 1 -!- 2. 

35. Show that 


V 

BitClI.l- 




(Here the sum is over all nonempty subsets of the set 
of the n smallest positive integers.) 

36. Use mathematical induction to show that given a set of 
n - 1 positive integers, none exceeding 2n, there is at 
least one integer in this set that divides another integer 
in the set. 

37. (Calculus required) Use mathematical induction to 
prove that the derivative of fix) = \* equals fi.U" 1 
whenever n is a positive integer. (For the inductive 
step, use the product rule for derivatives.) 

38. Suppose that 

a 0 ] 

0 ij 

where a and b are real numbers. Show that 


A - 


PDF created with pdfFactory trial version www.pdffactorv.com 





3.2 Exercises 201 


for ever}' positive integer n. 

39. Suppose that A and B are square matrices with the 
property AB = BA. Show that AB” = B\4 for every 
positive integer n. 

40. Suppose that m is a positive integer. Use mathemati¬ 
cal induction to prove that if a and b are integers with 
a = ft (mod m), then a* = h f: (mod m) whenever k is a 
nonnegative integer. 

41. Use mathematical induction to show' that if Aj, Aj. 

A„ and B tire sets, then 

(A) U A 2 U "■UAjnfi 

= {A - t n B) u (A z n B) u ■ ■ ■ u (A„ n £). 

42. Prove that if Ai, A;_.4,, and fij, B-,. ., B„ are sets 

such that Aj C Bi for k = 1,2. n, then 

a) [J 3* C (J S; : . b) A* c f| B k . 

t»i * i * - ] * = i 

43. Use mathematical induction to prove that if A i, A;,, 
A., are subsets of a universal set U, then 

l> = C A - 

; = i i * i 

44. Use mathematical induction to show that 
—i( p i v P 2 V • • • V Pn) is equivalent to -i p s A 
-i/jj A '--A -p, whenever p ], p 2 . p« are 
propositions, 

*45. Show that 

[ip\ A (/?„_, — p n )J 

tcpi Ap ; A--' A Pll ..{) — p„] 

is a tautology whenever pj, pz, .... p„ are propositions. 

46. Use the formula for the sum of the terms of a geometric 
progression to evaluate the following sums. 

a) 4 + 4 • 3 + 4 • 3 2 + ■ - - + 4 ■ 3 # 

b) 3 + 3 ■ 2 1 + 3 ■ T t • ■ ■ + 3 ■ 2 U1 

c) 1 - 2 + 2 3 - 2 1 + ■■■ + (- 1)*2* 

47. What is wrong with the following ‘‘proof" that all 
horses are the same color? 

l et Pin) be the proposition that ail the horses in 
a set of n horses are the same color. Clearly. P(l) 
is true. Now assume that Pin) is true, so that all 
the horses in any set of n horses are the same color 
Consider any n + 1 horses; number these as horses 
1,2.3....... fl-r 1, Now' the first n of these horses 

all must have the same color, and the last n of these 
must also have the same color. Since the set of the 
first n horses and the set of th e last n horses Overlap, 
all n ■ 1 must he the same color. This shows that 
Pin r l) is true and finishes the proof by induction, 


*48. Find the flaw with the following "proof” that a" - 1 
for all nonnegative integers n, whenever a is a nonzero 
real number, 

BASIS STEP: a a = 1 is true by the definition of a l! , 
INDUCTIVE STEP: Assume that a* = I for all non- 
negative integers k with H n. Then note that 
_ tJ n ' a” _ j_J_ = ( 
a n 1 1 

*49. Show that the second form of mathematical induction 
is a valid method of proof by showing that it follows 
from die well-ordering property, 

*50. Show that the following form of mathematical induc¬ 
tion is a valid method to prove that Pin ) is true for all 
positive integers n. 

BASIS STEP: P(\) and P( 2) are true. 

INDUCTIVE STEP: For each positive integer n, if Pi n) 
and Pi n +- 1) are both true, then P{n + 2) is true. 

In Exercises 51 and 52, H„ denotes the nth harmonic 
number, 

*51. Use mathematical induction to show that situ 
w'hcncver n is a nonnegative integer. 

*52. Use mathematical induction to prove that 

H\ t H-i * • ■ • + H„ = (n + 1 )H„ - n 
*53. Prove that 

, 11 1 „ 

1 + ---+—+■■■+ — > 2 Wn + 1 - I). 
v /2 y'3 ,/n 

*54. Show that n lines separate the plane into in 2 t ft + 2)/2 
regions if no two of these lines are parallel and no three 
pass through a common point. 

**55. Let tii, di. _ a„ be positive real n uni hers '[’he arith¬ 

metic mean of these numbers is defined by 

A - Ai + aj + - ■ ■ + a„)/n, 
and the geometric mean of these numbers is defined 
by 

G = (a t a2---a„) ]l ’‘. 

Use mathematical induction to prove that A jfc G. 

*56. Use mathematical induction to show that 21 divides 
4" +1 - 5 1 ” -1 whenever n is a positive integer. 

57. Use mathematical induction to prove Lemma 2 
of Section 2.5, which states thaL Up is a prime 
and p | rqaj where a , is an integer for 

i = 1,2, 3. n then /; | a; for some integer r. 

*58. The well-ordering property can be used to show that 
there is a unique greatest common divisor of two posi¬ 
tive integers. Let a and b be positive integers, and let S 
be the set of positive integers of the form as + bt, where 
s and t are integers. 

a) Show that $ is nonempty, 

b) Use the well-ordering property to show that S has a 
smallest element c, 
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Cl Show that if d is a common divisor of a and b, then 
d is a divisor of c. 

d) Show that c \ ti and t f b. (Hint: First, assume that 
t' l'a. Then a - <)c + r, where 0 < r < t. Show 
that r G S, contradicting the choice of c.) 
e> Conclude from (c) and (d) that the greatest com¬ 
mon divisor of a and b exists. Finish the proof by 
showing that this greatest common divisor of two 
positive integers is unique. 

*59, Show that if H|, a-i . a,-, are n distinct real numbers, 

exactly n ~ 1 multiplications are used to compute the 
product of these n numbers no matter how parentheses 
are inserted into their product. {Hint: Use the second 
principle of mathematicEil induction and consider the 
last multiplication.) 

60, Construct a tiling using L-shaped pieces of the 4 x 4 
chessboard with the square in the upper left comer re¬ 
moved, 

61, Construct a tiling using L-shaped pieces of the 8x8 
chessboard with the square in the upper left comer re¬ 
moved. 

62, Prove or disprove that all chessboards of the follow¬ 
ing shapes can be completely covered using L-shaped 
pieces whenever n is a positive integer. 


a) 3 x 2" b) 6 X 2" 
c) 3" x 3" d) 6" x 6* 

*63. Show that a three-dimensional 2" x 2* x 2" chessboard 
with one 1 x 1 x 1 cube missing can be completely 
covered by 2 x 2 X 2 cubes with one l x 1 x 1 cube 
removed, 

*64. Show that an n X n chessboard with one square removed 
can be completely covered using L-shaped pieces if 
n > 5, n is odd, and ft is not divisible by 3. 

65. Show that a 5 x 5 chessboard with a comer square re¬ 
moved can be tiled using L-shaped pieces. 

*66. Find a 5 X 5 chessboard with a square removed that 
cannot be tiled using L-shaped pieces. Prove that such 
a tiling does not exist for this board, 

67, Let a be an integer and d be- a positive integer. Show 
that ihc integers q and r with a = dq + r and 0 £ r < 
d, v/hich were shown to exist in Example 1, are unique. 

& 68, Use the principle of mathematical induction to show 
that P(n) is true for n = k, k + 1, k. + 2,.. , where k is 
an integer, if P(k) is true and the implication Pi n ) ^ 
Pin + 1) is true for all positive integers n with n > k. 

**69, Can you use the well-ordering property to prove the fol¬ 
lowing statement? “Every 1 positive integer can be de¬ 
scribed using no more than 15 English words"? 


33 _ 

Recursive Definitions 

INTRODUCTION 


Sometimes it is difficult to deft neat! object explicitly. However, it may be easy to define 
this object in terms of itself. This process is called recursion. For instance, the picture 
shown in Figure 1 is produced recursively. First, an onginal picture is given. Then a 
process of successively superimposing centered smaller pictures on top of the previous 
pictures is carried out. 

We can use recursion to define sequences, functions, and sets. In previous discus¬ 
sions, we specified the terms of a sequence using an explicit formula. For instance, the 
sequence of powers of 2 is given by a n = 2" for n = 0, 1, 2.However, this se¬ 

quence can also be defined by giving the first term of the sequence, namely, a 0 = 1, 
and a rule for finding a term of the sequence from the previous one, namely, a n+ i = 2a„ 
for n = 0,1,2_ 

RECURSIVELY DEFINED FUNCTIONS 


To define a function with the set of nonnegative integers as its domain, 

L Specify the value of the function at zero. 

2. Give a rule for finding its value as an integer from its values at smaller integers. 
Such a definition is called a recursive or inductive definition. 
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EXAMPLE I 


EXAMPLE 2 



FIGURE 1 A Recursively Defined Picture. 


V 


Suppose that/ is defined recursively by 
/(0) = 3, 

/Cm + 1 ) = 2 f{n) + 3. 

Find /(l). f( 2). /(3), and /(4). 

Solution; From the recursive definition it follows that 
/Cl) = 2/(0) 4-3 = 2-3 + 3 — 9, 

/(2) = 2/(1) + 3 = 2*9 + 3-21, 

/(3) - 2/(2)+ 3 = 2-21 - 3 = 45, 

/(4) = 2/(3) + 3 = 2-45 + 3 = 93, 


Many functions can be studied using their recursive definitions. The factorial func¬ 
tion is one such example. 


Give an inductive definition of the factorial function F(n) = nl. 

Solution: We can define the factorial function by specifying the initial value of this 
function, namely. F(0) - 1. and giving a rule for finding Fin + 1) from F(n). This is 
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iwvim 4 


EXAMPLE 4 


obtained by noting that (n -r 1 >! is computed from «! by multiplying by n 4 1. Hence, 
the desired rule is 

F{n t 1) = (n t 1 )F(n). B 


To determine a value of (he factorial function, such as F{5) — 51, from the recursive 
definition found in Example 2, it is necessary to use the rule that shows how to express 
Fin + 1) in terms of F(n) several times: 

F(5) = 5/ (4. - 5 • 4/ i3) = 5 ■ 4 ■ 3F(2) 5 • 4 • 4 IF- 1) 

- 5 ■ 4 - 3 ■ 2 ■ 1 ■ F( 0) = 5 ■ 4 ■ 3 • 2 ■ 1 ■ 1 = 120. 

Once F(0.j is the only value of the function that occurs, no more reductions are neces¬ 
sary. The only thing left to do is to insert the value of F tO) into the formula, 

Recursively defined functions are well-defined. This is a consequence of the prin¬ 
ciple of mathematical induction. (See Exercise 44 at the end of this section.) Additional 
examples of recursive definitions arc given in the following examples. 


Give a recursive definition of tr’ J where a is a nonzero real number and n is a tionnegafi ve 
integer. 

Solution: The recursive definition contains two parts. Eirst a 0 is specified, namely, 
ei u - I. Then the rule for finding « fl+1 from A namely, A 1 - a tf, for n = 
(f 1,2,3,..., is given. These two equations uniquely define a" for all nonnegative 
integers n. ■ 


Give a recursive definition of 



t - o 


Solution: The first pan of the recursive definition is 

i.i 

'T Uk — Ufj. 
k (! 

The second part js 

k--o ,- u / * 

hi some recursive definitions of functions, the values of the function at the first 
k positive integers arc specified, and a rule is given for determining the value of the 
function at larger integers from its values at some or all of the preceding k integers. 
That such definitions produce well-defined functions follows from the second principle 
of mathematical induction (sec Exercise 45 at the end of this section). 
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EXAMPLE 5 The Fibonacci numbers, f Q , f\ ,f 2 . are defined by the equations fa = 0. J] - 1, 

and 


web A ~ fn-) + fn-2 

for n = 2, 3,4,,... What are the Fibonacci numbers /?, f 3 , f 4 , f\. / fi ? 

Solution: Since the first part of the definition states that fa = Oand/i = I, it follows 
from the second part of the definition that 

ft '= /, + /o =1+0-1, 
h ■= .h - f - 11 

A = h + ft - 2 + 1 - 3 , 

h - U + h -3 + 2 = 5 , 

ft - /5 - A =5 + 3 = 8 . a 


We can use the recursive definition of the Fibonacci numbers to prove many prop¬ 
erties of these numbers. We give one such property in the next example. 


EXAMPLE 6 Show that /„ > a" where a = (1 + v"5)/2, whenever n 3. 

Solution: We can use the second principle of mathematical induction to prove this in¬ 
equality, Let P{n) be the statement f n > a"~ 2 . We want to shew that P(n) is true when¬ 
ever n is an integer greater than or equal to 3. 

First, note that 

a < 2 = J 3 , a 2 = (3 -r v' 5 )/2 < 3 - A, 

so that P{ 3) and P(4) are true. Now assume that P(k) is true, namely, that / t > a k : , 
for all integers k with 3 £ i £ n, where n = 4, We must show that P\n \ 1) is true, 
that is, that f„ +] > a" K Since a is a solution of x 2 - x I = 0 fas the quadratic 
formula shows), it follows that a : = a + I, Therefore, 

a" 1 = a‘ • at* 2 = (a 4 Dot 1 '”' - a - a" 5 - 1 • tt""' 1 ~ a* 2 t a’ r "\ 

By the inductive hypothesis, if n 5, it follows that 

fn- ]>£*"'’. f n >a n2 

Therefore, we have 

- fn + fn [ > f»" 2 + a"' 2 ’ = a” 

It follows that P{n i- 1) is true. This completes the proof. ■ 


Fibonacci te 11 HO-1228;. Fibonacci (short totfilius Sonant or "son ot Bonacci”) was also known as 
Web Feonardo of Pisa. He was bom in the Italian commercial center nf Pisa. Fibonacci was a merchant who trav¬ 
eled ex lensivdy throughout the Mideast, where he came into contact with Arabian mathematics, in his book 
Liber Abaci Fibonacci introduced the European world to Arabic notation for numerals and algorithms for 
arithmetic. It was in this book that his famous rabbit problem (described in Section 5.1) appeared. Fibonacci 
also wrote bonks on geometry and trigonometry and on Diophantine equations, which involve finding integer 
solutions to equations, 
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THEOREM 1 


Remark: The inductive step shows that whenever na4, Pin + i) follows from the 
assumption that P(k) is true for 3 s t i n. Hence, the inductive step does not show 
that P(Y) —> F{4), Therefore, we had to show' that F(4) is tree separately. 

We can now show that the Euclidean algorithm uses 0(log b) divisions to find the 
greatest common divisor of the positive integers a and b, where a > b. 

LAME’S THEOREM Let a and b be positive integers with a ^ b. Then the num¬ 
ber of divisions used by the Euclidean algorithm to find gcd(o» b) is less than or equal 
to five times the nu m ber of decimal digits in b. 


Proof: Recall that when the Euclidean algorithm is applied to find ged(a, b) with a s 
b, the following sequence of equations f where a = and b = rj) is obtained. 

rg = f|tji + ri 0 ^ r 2 < r\ 

f[ = riqi t n Os rj < rj 


r„- 2 = r n -[(j n i +r„ 0</„< r„. [ 

r, i -1 = r„q„ 

Here n divisions have been used to find r„ = gcd(a, b). Note that the quotients 

q i,f /2 . q„ i are tdl at least 1. Moreover, q n ' 2, since r n < r„_]. This implies 

that 

r„ a 1 = f h 
b i ^ 2r„ > 2fi = fi, 
r n-2 - r n 1+6 5 + h ~ /+- 


n ^ + r 4 > j n _ l -h j n _ 2 - j n , 

b ~ r i ri + n - fa + /„ -1 - /„ - \. 

It follows that if n divisions are used by the Euclidean algorithm to find ged(a, b ) with 
a ?■ b, then b > /„+[. From Example 6 we know that / B+ j > a" -1 for n > 2, where 
a — (1 + v'5)/2. Therefore, it follows that b > a" -1 . Furthennore, since log !0 « ~ 
0.208 > 1/5, we see that 

logic b > (n - I)log I( ,ffl > (n - 1 )/5. 

Hence, n- 1 < 5 ■ log 10 h. Now suppose that b has A" decimal digits. Then b < I t) 1 and 
logic i}< -It follows that n - 1 < 5 k, and since k is an integer, it follows that n - ■ 5k. 
This finishes the proof. □ 

Since the number of decimal digits in b. which equals Jlog 10 b\ + 1, is less than 
or equal to log po b + 1, Theorem 1 tells us that the number of divisions required to 
find gedtu, b) with a > h is less than or equal to 5(log 10 fc + 1), Since 5(!og K .F + 1) 
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EXAMPLE 7 


mb 


is 0(Iog b), we see that O(\ogb) divisions are used by the Euclidean.'algorithm to find 
gcdlfl, b) whenever a > b. 

RECURSIVELY DEFINED SETS 


Recursive definitions are often used to define sets. When this is done, an initial col¬ 
lection of elements is given. Then the rules used to construct elements of the set from 
other elements already known to be in the set are given. Sets described in this way are 
well-defined, and theorems about them can be proved using their recursive definitions. 
Some examples of recursive definitions of sets follow. 


Let S be defined recursively by 
3 £ 5; 

x + y £ S if x E S and v £ 5. 

Show that S is the set of positive integers divisible by 3. (Note that implicit in this 
definition is the assumption that nothing belongs to S unless it can be generated using 
the two statements in the recursive definition of S.) 

Solution: Let ,4 be the set of all positi ve integers divisible by 3. To prove that A = S, 
we must show that A is a subset of S and that A is a subset of A. To prove that A is a 
subset of S, we must show that every positive integer divisible by 3 is in S. We will use 
mathematical induction to prove this. 

Let Fin) be the statement that 3 n belongs to S. The basis step holds since, by the 
first pan of the recursive definition of S, 3 X 1 = 3 is in S, To establish the inductive 
step, assume that Pin) is true, namely, that 3 n is in A, Since in is in S and since 3 is in 
5, it follows from the second part of the recursive definition of S that 3n + 3 = 3(n + 1) 
is also in S. 

To prove that Sis a subset of A, we use the recursive definition of S. First, the basis 
step of the definition specifies that 3 is in S. Since 3 = 3x1, all elements specified to 
be in S in this step are divisible by 3. To finish the proof, we must show T that all integers 
in S generated using the second part of the recursive definition are in A. This consists 
of showing that a 1 + y is in A whenever x and y are elements of A also assumed to be in 
A. Now if a and y are both in A, it follows that 3 | a and 3 | y. By Theorem 1 of Section 
2,3, it follows that 3 | x + y. completing the proof. ■ 


Gabriel Lame (1795-1870). Gabriel Lame entered the Ecole Poly technique in 1813, graduating in 1817. 
He continued his education at the Ecole des Mines, graduating tn 1820. 

In 1820 Lame went to Russia, where he was appointed director of the Schools of Highways and Trans¬ 
portation in St Petersburg. Not only did he teach, hut he also planned roads and bridges while in Russia. 
He returned to Paris in 1832, where he helped found an engineering firm However, he soon left the firm, 
accepting the chair of physics at the Ecole Pelytechnique, which he held until 1844, ’While holding this 
position, he was active outside academia as an engineering consultant, serving as chief engineer of mines 
and participating in the building of railways. 

Lame contributed original work to number theory, applied mathematics, and thermodynamics. His 
best-known work involves the introduction of curvilinear coordinates. His work on number theory includes 
prov ing Fermat s Last Theorem for n = 7, as well as providing the upper bound for the number of divisions 
used by the Euclidean algorithm given in this text, 

fn the opinion of Gauss, one of the most important mathematicians of all time, Lame was the foremost 
French mathematician of his time. However, French mathematicians considered him too practical, whereas 
French scientists considered him too theoretical, 
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The recursive definition of a set in Example 7 is typical. First, an initial set of 
elements is given. Second, a rule is provided for generating new elements from those 
already known to be in the set. Implicit in the definition is that no element belongs to 
the set unless it is listed in the initial set of elements or it can be generated using the 
rule given for constructing new elements. 

One of the most common uses of recursive definitions for sets is to define well- 
formed formulae in various systems. This is illustrated in the following examples. 

EXAMPLE 8 

The well-formed formulae of variables, numerals, and operators from {+, *,/, f} 

(where * denotes multiplication and j denotes exponentiation) are defined by 

a; is a well-formed formula if a is a numeral or a variable; 

(/ ~ g)i {/ - .?), (/ * g), [fig), and (/ t g) are well-formed formulae if/ and g 
are 

For instance, from this definition, since x and 3 are well-formed formulae, {x + 3), 
(jc - 3), (x * 3), fx/3), and (x f 3) are well-formed formulae. Continuing, since v also 
is a well-formed formula, so are ((x + 3) + y), (y - (x * 3)), and so on. (Note that (3/0) 
is a well-formed formula, since only syntax matters here.) ■ 

EXAMPLE 9 

The well-formed formulae for compound propositions involving T, F, propositional 
variables, and operators {-„ A, V* are defined by 

T, E, and p, where p is a propositional variable, are well-formed formulae; (-i p), 

(p y q). (p/\ q). (p -* q), (p q) are well-formed formulae if p and q are well- 
formed formulae. 

For example, if p t q, and r are propositional variables, then repeatedly using the recur¬ 
sive definition shows that (p \J q), (r A T). and ((/> y q) — (r A T)) are well-formed 
formulae. ■ 

Recursive definitions are often used m the study of strings. Recall from Chap¬ 
ter 1 that a string over an alphabet 2 is a finite sequence of symbols from 2. The 
set of Strings over 2 is denoted by 2*. Two strings can be combined via the operation of 
concatenation. The concatenation of the strings x and y, denoted by x v. is the string x 
followed by the stringy. For instance, the concatenation of x = ahra and y = cadabra 
is xv = abracadabra. The following recursive definition is often used when results 
about strings are proved. 

EXAMPLE 10 

A Recursive Definition of the Set of Strings The set 2’ of strings over the alphabet 

2 can be defined recursively by A E 22 where A is the empty string containing no 
symbols, and wx £ 2’ whenever w E 2* and x E 2. 

The first pan of this definition says that the empty string belongs to 2*. The second 
part states that new strings tire produced by concatenating strings in 2’ with symbols 
from 2. ■ 


The length of a string, which is the number of symbols in the siring, can also be 
defined recursively. 
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EXAMPLE 11 Give a recursive definition of i(w), the length of the string w. 

Solution: Tlie length of a string can be defined by 
K A) = 0; 

l{wx) - Hw) + 1 if w e T' and .tel ■ 

The following example illustrates how the recursive definition of strings can be 
used in proofs. 


EXAMPLE 12 Use mathematical induction to prove that /(xv) = /lx) 4- I(y), where a and v belong to 

2'. the set of strings over the alphabet 2. 

Solution: We will base our proof on the recursive definition of the set 2* given in Exam¬ 
ple 10. Let Ply) be the statement that Hxy) ~ lix) + /(v) whenever x belongs to 2\ 
BASIS STEP: To complete the basis step, we must show that P( A) is true. That is, 
we must show that l(xk) = /(jc)+/(A) forallx E 2*. Since t(xA) = l(x) - /(;> • () = 
Hx) + /(A) for every string x, it follows that P(A) is true, 

INDUCTIVE STEP: To complete the inductive step, we assume that Pi v) is true 
and show that this implies that P(\a) is true whenever a £ 2. What we need to show is 
that l(xya) - l(x) + l(ya ) for every a £ 2. To show this, note that by the recursive defi¬ 
nition of J(w) (given in Example 11), we have Hxya) = /(xv) + 1 and l{ya) = l{\-) 4 ], 
And, by the inductive hypothesis, Hxy) = l(x) + t{y). We conclude that l(xya) = 
!(x) + l(y) + 1 - /(x) + I(yu), m 


Exercises 

1. find f\ I). / [2i, /(.)!, and /(4j if fui] is defined 
recursively b> /(0) -- 1 and for 11 = 0 1,2, 

a) f(n + I) fin) + 2 

b) fin ■ 1) = 3/(ft). 

c) f(n + 1) = 2 ,l ' l) . 

dl fin + 1) ■■ Jinf r fin) + I. 

2. Find /(I), f(2). /(?). /J4J, and /(5) if f(n) 
is defined recursively by /ft)) = 3 and for n 

0. 1.2, ... 

ill or. ■¥ 1 j -• -2/fn I. 
bi fit< -■ 1) ... 3j\n) : 7. 

c) fin + I) - /<«>- - 2fin) ~ 2 . 

d) fin + 1) - 37 

.1. Pi ml f(7). /(. 1 ), /(4 J. and /(5) if / is defined 
recursively by /(0) - -I, /(]) = 2 and for n 
= 1 , 2 . ... 

a! fui ?- 1 .1 ■■ f{n) j- "ifin - I). 

hi fin 1 2 I 1 = fUiVfin i). 

rt j in t I i - If inf ~ 4 ft tt - ] 

d .1 / t« - 1 1 = sin ■■■ 1 i i fui) 


4. Find /(2), /(3). /(4). and /(Si if / is defined 
recursively by /(0) -- /tit - I and for rt ■ 
1.2,... 

aj f{n + 1) = fin) - Jin - !). 

b) i! » ■ 1 : - mm- 1 ). 

c) fin + I) = f(nf + fin - 1)1 

d) fin + 1) = fin)/fin - I), 

5. Give a recursive definition of the sequence pi,,}. 
n - 1,13, ...if 

a) = 6u. b) a ,, 2 it I. 

c) a„ = 10". d) u„ = 5. 

6. Give a recursive definition of the sequence {a n }, 
n = 1,2,3,... if 

a) a,. 4n - 2. b) a„ = ! t (- I)" 

c) a„ = n(n + 1). d) a„ = n ; . 

1. Lei F be (be function such that Fin) is the sum of 
the first n positive integers. Give a recursive defi¬ 
nition nf Fin' l. 

8. Give a recursive definition of S„.in). the sum of the 
integer m and the nonnessative integer;;. 
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9. Give a recursive definition of P m (n), the product of 
the integer m and the non negative integer n. 

In Exercises 10-17 f„ is the nth Fibonacci number. 

10, Prove that fa + fa + — f fa - /„/ B+1 whenever 
n is a positive integer, 

11. Prove that /i + f, + ■ ■ ■ + /?„- r =■ /?„ whenever n 
is a positive integer, 

*12, Show that /„ t ] f„ i - fa - (-1)" whenever n is a 
positive integer. 

*1.3, Show that fa fi ~f x /: + ■■■ + f 2 „- 1 fa„ - fa„ when¬ 
ever a is a positive integer. 

*14. Show that fa-J]+Js - ha-i+fa, = fa« i~l 

whenever n is a positive integer. 

15. Determine the number of divisions used by the Eu¬ 
clidean algorithm to find the greatest common divi¬ 
sor of ihe Fibonacci numbers /„ and /„, [ where n 
is a nonnegative integer. Verify your answer using 
mathematical induction, 

16. Let 


Show that 


whenever n is a positive integer. 

17 . By taking determinants of both sides of the equa¬ 
tion in Exercise 16, prove the identity given in Ex¬ 
ercise 12. (This exercise depends on the notion of 
the determinant of a 2 X 2 matrix.) 

* 18 . Give a recursive definition of the functions 
max and min so that maxfeii, a;,..., a*) and 
min(m, a 2 _,«„) are the maximum and min¬ 

imum of (he n. numbers at, a 2r ..., a„> respec¬ 
tively, 

*19. Let tij, a 2 . ., a„, and h\, hi, _ h„ be real num¬ 

bers. Use the recursive definitions that you gave in 
Exercise 18 to prove the following. 

a) maxi ■ -rq, ■■ a — -a„)=- uj, _ a n ) 

b) max(tt[ + h], ci; t lx,_ a„ + b a ) 

x maslai.u;,. ..,a n ) + max(h[,hi. b„) 

c) min(ci i + h u a> + hj_ ,a„ + b„) 

-a tninffl,, a;_ g n ) + bi . lt„) 

20. Show that the set V defined by 1 E S and a tie 
.S' whenever s E S and t E 5 is the set of positive 
integers 

21. Give a recursive definition of ihe set of positive in¬ 
tegers that are multiples of 5. 

22. Give a recursive definition of 

a) the set of odd positive integers. 

b) the set of positive integer powers of 3. 

e) the set of polynomials with integer coefficients. 


23. Give a recursive definition of 

a) the set of even integers. 

b) the set of positive integers congruent to 2 mod¬ 
ulo 3. 

e) the set of positive integers not divisible by 5. 

24. Show that any well-formed formula of numerals, 
variables, and operators from { + t] contains 
the same number of right and left parentheses. 

25. Define well-formed formulae of sets, variables rep¬ 
resenting sets, and operators from { . U, fl, - |. 

The reversal of a string is the string consisting of the sym¬ 
bols of the string in reverse order. The reversal of the string 
w is denoted by w K . 

26. Find the reversal of the following bit strings. 

a) 0101 

b) 1 10H 

c) 100010010111 

27. Give a recursive definition of the reversal of a 
string. {Hint: First define the reversal of the empty 
string. Then write a string vv of length n + I as cry, 
where x is a string of length n, and express the re¬ 
versal of w in terms of x R and y.) 

*28. Give a recursive proof that (wi wj ) K = wfi-vf. 

29. Give a recursive definition of w 1 where w : is a string 
and i is a nonnegative integer. (Here w' represents 
the concatenation oft copies of the string w.) 

*30. Give a recursive definition of the set of bit strings 
that are palindromes. 

31. When does a string belong to the set A of bit strings 
defined recursively by 

A E 4 

O.d 6 A if x E A. 

where A is the empty string? 

*32, Recursively define the set of bit strings that have 
more 0s than Is. 

33. Use Exercise 29 and mathematical induction to 
show that i(vri) = i ■ /(>v), where w is a string and 
i is a non negative integer. 

*34, Show that (tv*)' = (V) fl whenever >v is a string 
and i is a nonnegative integer; that is, show that the 
ith power oT the reversal of a string is the reversal 
of the ith power of the string. 

*35. A partition of a positive integer n is a way to write 
n as a sum of positive integers. For instance, 7 - 
3 f 2 + 1 + 1 is a partition of 7, Let P m equal the 
number of different partitions of m, where the order 
of terms in the sum does not matter, and let P mj] be 
the number of different ways to express m as the 
sum of positive integers not exceeding n, 

a) Show that P„ im = P m . 

b) Show' that the following recursive definition for 
Pm ,.i is correct: 
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wtb 


1 

1 

P mm 

I + /V- l 
P m ,„-\ + 


if m = 1 
if n = 1 
if m < n 
if m — n > I 
if m > n > 1 


c) Find ihe number of parti (ions of 5 and of 6 using 
this recursive definition. 

Consider the following inductive definition of a version 
of Ackerman iris function. This function was named after 
Wilhelm Aekermann, a German mathematician who was 
a student of the great mathematician David Hilbert. Ack- 
ermann's function plays an important rote in the theory of 
recursive functions and in the study of ihe complexity of 
certain algorithms involving set unions. (There are several 
different variants of this function. All are called Acker- 
martn’s function and have similar properties even though 


their values do not always agree.) 

usual. The function log* n 


" 2n 

if tn - 0 

r* 


0 

if m s l and n -■ 0 

lo g (it ft = !,)g(log ^ 

Aim. n) = < 

2 

if m Sr 1 and n = 1 


A[m - 

1, A(ffi, n — 1)) 

l undefined 


if m a 1 and n -^ 2 
Exercises 36 to 43 involve this version of Aekermann's 
function. 


d) F(n) - 1 + F(n/2) if n is even and n & 2, 
F(n) = 1-F(n- 1) if n is odd, and F(l) - 1. 

e) F(n) - 1 +Fin/2) if n even and n s 2 ,F(n) — 
F(3n - 1) if n is odd and n s 3, and F{1) = 1, 

47. Show th ai each of the following proposed recursive 
definitions of a function on the set of positive inte¬ 
gers does not produce a well-defined function. 

a) F(n} = 1 + F([(n + [ )F1 \) for n =s 1 and 

F{ 1 ) = 1 - 

b) Fin) = 1 + Fit i - 2) for n > 2 and F( 1) - 0. 
ci F{n) = I f F(nf 3) for n s 3,F(1) = 1. 

F(2) = 2, and F(3) = 3. 

d) F(nj - 1 + F(nFl) if n is even and n 5* 2. 
Frii) = I + Fin - 2) if n is odd, and F(l) =- 1. 

e) Fin) = 1 +F(F(n-1)) if n 2 2andF(1) = 2. 

Exercises 48-51} deal with iterations of the logarithm func¬ 
tion. Let log n denote the logarithm of n to the base 2, as 
ined recursively by 

if k = 0 

if log 1 '* -1 ' n is defined 
and positive 
otherwise. 

The iterated logarithm is the function log* n whose 
value at n is the smallest nonnegalive integer k such that 

log* 71 1. 


36. Find the following values of Aekermann’s function, 
a) .4(1,0) b) Aif, \) 

C) A( 1, I) d) A12,21 

37. Show' that A(m, 2) = 4 whenever ms i. 

38. Show that At l,7i) = 2" whenever n > ]. 

39. Find the following values of Ackennann’s function, 

a) ,4(2, 3) *b) A(3, 3) 

*411. Find ,4(3,4). 

**41. Prove that A(m, n + l) > A(m, fi) whenever m and 
a are nonnegative integers, 

*42. Prove that A(rn + i, n) s Aim, n) whenever m and 
n are nonnegative integers. 

43. Prove that A(i,j) s j whenever i and j are non- 
negative integers, 

t 44. Use mathematical induction to prove that a function 
F defined by specifying F(0) and a rule for obtain¬ 
ing fin -e I) from Fin) is well-defined. 
f 45. Use the second principle of mathematical induc¬ 
tion to prove that a function F defined by specify¬ 
ing F (0) and a rule for obtaining fin + I) from the 

values F{k) for k == 0, 1,2__ it is well-defined. 

46. Show that each of the following proposed recursive 
definitions of a function on the set of positive inte¬ 
gers does not produce a well-defined function. 

a) Fin) = I F( l re/2 J) for n ? 1 aridF(l) = I. 

b) Fin) --- 1 + Fin - 3) tor n • 2, F(l) =* 2. and 
F(2) - 3. 

cl F(n) “ 1 + F(o/2) lor n s 2 , Ffl) =-- 1, and 
F(2) = 2. 


48. Find each of the following values: 

a) log* 16 b) log* 256 

G log* 2 M -™ d) log 14 ’ 2 lbim 

49. Find the value of log" n for each of the following 
values of tt: 

a) 2 b> 4 c) 8 d) 16 

el 256 f) 65536 g) 2 2 '™ 

50. Find the largest integer n such that log* n = 5. De¬ 
termine the number of decimal digits in this number. 


Exercises51 -53 deal with values of iterated functions. Sup¬ 
pose that /(n) is a function from the set of real numbers, or 
positive real numbers, or some other set of real numbers, to 
the set of real numbers such that fin) is manotonicalfy in¬ 
creasing [that is, fin) < fim) when n < m) and fin) < tt 
for all n in the domain of/.] The function is defined 
recursively by 


/'*(«) 


fn if k = 0 

')(«)) if A -0 


Furthermore, let c be a positive real number The iterated 
function f' is the number of iterations of / required to re¬ 
duce its argument to c or less, so that f‘(n ) is the smallest 
nonnegative integer k such that f k (n) - c. 

51. Let /"( 77 } = n~a, where n is a positive integer. Find 
a formula for f {k) (n). What is the value of f'in) 
when n is a positive integer? 

52. Let f{n) = nil. Find a fonnula for f a, (n). What 
is the value of f'(n) when n is a positive integer? 
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53. 1 .cl. fin) - ^ n. Find a formula for What 

is ttie value of ffin) when n is a positive integer'. 1 ’ 

Exercises 54-ftl deal with some unusual sequences, in¬ 
formally called seif-gene rating sequences, produced by 
simple recurrence relations or rules. In particular, Fa- 
ercises 34-37 deal with the sequence :(«(»)} defined by 
j(/?l ■■ tt - (Haiti 1) ) for n 1 and h(0) = 0. (This 
sequence, as well as those in Exercises 58 and 59, are 
ddined in Douglas Hcsfstadcr's fascinating book Codel, 
F.wiifr. Bach ([Ho99|). 

54. Find the first 10 terms of the sequence {«(«}} de¬ 
fined in the preamble to this exercise. 

*55, Prove that this sequence is well-defined, That is, 
show that (dn) is uniquely defined for all nonnega¬ 
tive integers n 

**56. Prove that ntn) = L (rt + lj/ij where p = (-1 -t 
v '5)/2. 'Jim;. First show tor ail n > 0 that (/an - 
\fin\) - lp J nj) - 1. Then show thrall real 


numbers« withO a; u < 1 and« ^ I ■ grthaqd + 

р) ( I - ii)j 1 [a * fi; = 1, considering the cases 
0 s' it ■ 1 - /.(, and i — pi < a -c i separately, j 

*57. Use the formula from Exercise 5(> to show that 
rtf ri) = «{« - I) if fui - [fiMj < I - p. and rtl.ii ) = 
a(n - 1) - 1 otherwise, 

58. Find the fi rst 10 te rms of each t ff the f yl to win g sc lt- 

generating sequences: 

a) tt{n) ■ n - hi a(a(n l))k for n te 1 with 
ofO) - 0. 

b) u(n) = n - (i(u(o(a(u - tot)) for n > 1 with 
fl(ft) = 0 . 

с) a(rt) - a(n - a(n - I )J f a(n - (tin -■ 21) for 
a > 3 with afl) = I and erf2J -- 1. 

59. Find the first 1(1 terms of truth the sequences mini 
and fin) defined by the following pair of interwo¬ 
ven recurrence relations: m(n) - n - /(m(>i - I fj, 

fin ) = n - mi fin - liifru/i c: 1 with /{()) - I 

and miO) -- 0. 


Godfrey Harold Hardy 1 1877-1947 1 . Hardy, bom in Cranleigh, Surrey, England, wars ihe older of two 
children of Isaac Hardy and Sophia Hall Hardy, His father was the geography and drawing master at the 
Cranleigh School and also gave singing lessons and played soccer. His mother gave piano lessons and helped 
ran a boardinghouse for young students. Hardy’s parents were devoted, to their children’s education. Hardy 
demonstrated his numerical ability at the early age of 2 when he began writing down numbers into the 
millions. He had a private mathematics tutor rather than attending regular classes at the Cranleigh School. 
He moved to Winchester College, a private high school, when be was 13 and was awarded a scholarship. 
He excelled in his studies and demonstrated a strong interest in mathematics. He entered Trinity College. 
Cambridge, in 1896 on a scholarship and won several prizes during his time there, graduating in 1899. 

Hardy held the position of lecturer in mathematics ai Trinity College at Cambridge l/lttwrsiiy front 
190fr lr> 1919, when he was appointed to the Sullivan chair of geometry at Oxford, He had become unhappy 
at Cambridge with the dismissal of the famous philosopher and mathematician Bertrand Russell from Trinity 
fur antiwar activities and did not like a heavy toad of administrative duties. In 193! he returned to Cambridge 
.» the Sadlerrian professor of pure mathematics, where he remained until Ins retirement in 1942. He was 
a pare mathematician and held an elitist view of mathematics, hoping his research could never be applied. 
Ironically, he is perhaps best known as one of the developers of the Hardv-Weinberg taw, which predicts 
patterns of inheritance. His work in this area appeared us a letter to the journal Science in which he used 
simple algebraic ideas to demonstrate emirs in an article on genetics. Hardy worked primarily ill number 
theory and function theory, working on such topics as the Riemann zeia function, Fourier senes, and the 
distribution of primes. He made many important contributions to many important problems, such as Waring s 
problem about representing positive integers as, the sum of kth powers and the problem of representing odd 
integers as the sum of three primes. Hardy is also remembered ffcir his collaborations with John E. Little wood, 
a colleague at Cambridge, with whom be wrote more than 100 papers and the famous Indian mathematical 
prodigy Srinivasa Ramanujan. His collaboration with Littlewood led to the joke that [here were on ly three 
important English mathematicians at that time. Hardy, Liitlewood. and Hurdv-l ,title wood, although some 
people thought that Hardy hud invented a fictitious person. Litllettnird, since L.ittlewood was seldom seen 
outside Cambridge, Hardy had the wisdom of recognizing Ramanujan’s genius from it neon vent emu] but 
extremely creative writings Ramanujan sent him. while other mathematicians failed to see the genius, Hardy 
brought Ramanujan to Cambridge and collaborated on important joint papers, establishing new resuh son the 
number of partitions nf an integer. Hardy was interested in mathematics education, and his book A Course 
in Pifc Mathematics bud a profound effect on undergraduate instruction in mathematics in the first half of 
the twentieth century. Hardy also wrote ,3 Mathematician it Apology in which he gives his answer tt' the 
question Whether it is worthwhile to devote one’s life to the study uf mathematics. It presents Hurds’s view 
■ll'cviiai i mu hematics w and what a mathematician dues. 

Hurds had a suvuig interest ill -.ports. He was an a-.ij cricket fan and followed scores Huse'iy One 
twculistr iia>: be had w a » thill be did :njt like ms picture taken mnly five snapshot me knmvn'ond JislTed 
minors, icvcciiig them with towel-, immediate.!', up,m entering a link’; arm ft 
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Golomh's self-generating sequence is the unique nondc- 
cleasing sequence of positive integers u\. hj, aj, ,.. which 
has tlie property that it contains exactly a^ occurrences of 
k for each positive integer k. 

fit), hind the first 20 terms of tiolomb s self-generating 
sequence. 

*61. Show that if /(h) is the largest integer m stich that 
u m = n, where a m is the /nth term of Gtttomb's 
self-generating sequence, then /Or) = , St 

ami /( /(h)) = vj =[ h a ^ 

The set d ul logarithmico-exponenlial functions, intro¬ 
duced by the famous British mathematician G. H. Hardy, is 
the smallest set of functions such that: 

• the function f(n J - « belongs to if. whenever a is a 
real number; 

• the function /(«) = n belongs to IT; 

• it the functions /(h) and gin) belong to if:, then fin) - 
|(./i) belongs to 

• if the function fin) belongs to if'., then e-' 1 "-' belongs 
to if'\ 

• it f inl belongs to .if and there exists an integer M such 


/(h) > 0 tor n he ,V (this means that f is called even¬ 
tually positive), then In /’(/it belongs to if, where in ,r 
denotes the natural logarithm of ,i. as usual. 

Hardy showed far every Jogarithmico-exponential function 
not identically zero that f(n) is eitlier eventually positive or 
eventually negative. He proved that if fin ) and gin) belong 
to .if. then either /(h) i s gift) is <>(/{ n )), or /! n I and 

gin) are of the same order. 

62. Show that if /(h ) and g(n) belong to if , then /( n t i 
g(/i) belong to if. 

*63, Show that if fin) and gin) belong to :£ and arc 
eventually positive, then /(n)g(/i) and /{nVgin} 
belong to if., and that this implies, using the fact 
that every iogarithmico-cxponenlial function is 
eventually positive, eventually negative, or identi¬ 
cally zero, that the product and quotient of any two 
functions, not identically zero, in if are in if. 

64. Use Exercises 62 and 63 to show that every poly¬ 
nomial f[nl = a m tl m + a m \n m ‘ + -*• a,j with 

real coefficients belongs to if.. 

65. Show that if f(n ) belongs to it and is eventually 
positive, then Jf{n) belongs to 'f. 

66. Show that the function e- "* 111 " ln ln " belongs to :■}. 


Srinriasa Ramanujan (1987-1920), The Famous mathematical prodigy Ramanujan was bom and raised 
in sou them India near the city o! Madras. His lather was a clerk in a ninth shop. His mother contributed to 
the family income by singing at a local temple. Ramanujan studied at the local English language school, 
displaying his talent and interest for mathematics. At 13 he mastered a textbook used by college s indent'. 
When he was 15. a university student lent him a copy of Svn(/p,fh of Pure Mathematics: Ramanujan decided 
to work out the ovot fi(XK) results in this book, slated without proof or explanation, writing on sheets Inter 
collected to fwm notebooks. He graduated from high school in 1904. winning a scholarship ip the Unfoersm 
ol Madras. Enrolling in a fine arts curriculum, he neglected his subjects other than mathematic;; and lost bis 
scholarship. He failed to pass examinations at the university four times from 1904 to 1907 , doing wcli o;;!\ 
in mathematics. During this time lie filled his notebooks with original writings, sometimes rediscovering' 
already published work and at other times making new discoveries. 

Without a university degree, it was difficult for Ramanujan to find a decent job. To survive, he had to 
depend on the goodwill of his inends. He tutored students in mathematics, but his unconventional ways of 
thinking and failure to stick to the syllabus caused problems. He was married in 1909 in an arranged marriage 
to a young woman 9 years his junior. Needing to support himself and his wife, he moved to Madras and 
sought a job. He showed hie notebooks of mathematical writings to his potential employers, but the books 
bewildered them. However, a professor at the Presidency College recognized his genius and supported him 
for a while, and in 1912 he found work as an accounts clerk, earning a small salary. 

Ramanujan continued his mathematical work during this time and published his first paper in 1910 tr: 
an Indian journal. He realized that his work was beyond that of Indian mathematicians and decided to write to 
leading English mathematicians. The first mathematicians he wrote to turned down his request for help. But 
in January 1913 he wrote to G. H. Hardy, who was inclined to turn Ramanujan down, but the mathematical 
statements in the letter, although stated without proof, puzzled Hardy, He decided to examine ihem doselv 
with the help of his colleague and collaborator J. E. Littlcwoud. They decided, after careful study, that 
Ramanujan was probably a genius, since his sialemcnti '’could only be written down by a mathematician 
of the highest class, they must be true, because if they were not true, no one would have the imagination ;n 
invent [herri." 

flardy arranged a scholarship for Rnmatiujan, bringing hint to. England in IPi-l. H-imv pafViulU 
ujluieii lurri in matheniuocul analysis, and diey eulluhoQletj for i years, proving siyiiijirant theorems .jhout 
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3.4 

Recursive Algorithms 


INTRODUCTION 

Sometimes we can reduce the solution to a problem with a particular set of input to the 
solution of the same problem with smaller input values. For instance, the problem of 
finding the greatest common divisor of two positive integers a and b where b> a can 
be reduced to finding the greatest common divisor of a pair of smaller integers, namely, 
b mod a and a, since gcdfh mod a, a) = gcdfo, h). When such a reduction can be done, 
the solution to the original problem can be found with a sequence of reductions, until 
the problem has been reduced to some initial case for which the solution is known. 
For instance, for finding the greatest common divisor, the reduction continues until the 
smaller of the two numbers is zero, since golf a, 0) = a when a > 0. 

We will see that algorithms that successively reduce a problem to the same problem 
with smaller input are used to solve a wide variety of problems. 


DEFINITION 1 . An algorithm is called recursive if it solves a problem by re¬ 
ducing it to an instance of the same problem with smaller input. 


We will describe several different recursive algorithms in the following examples. The 
first example shows how a recursive algorithm can be constructed to evaluate a function 
from its recursive definition. 


EXAMPLE 1 Give a recursive algorithm for computing a n where a is a nonzero real number and n is 

a nonnegative integer. 

Solution: We can base a recursive algorithm on the recursive definition of a a . This 
definition states that a" +1 = a • a" for n> 0 and the initial condition a u = 1. To find 
a”, successively use the recursive condition to reduce the exponent until it becomes 
zero. We give this procedure in Algorithm 1, ■ 


the number of partitions of integers. During this time, Ramanujan made- important contributions to number 
theory and also worked on continued fractions, infinite series, and elliptic functions. Ramanujan had amaz¬ 
ing insight involving certain types of functions and senes, but his purported theorems on prime numbers 
were often wrong, illustrating his vague idea of what constitutes a correct proof. He was one of the youngest 
members ever appointed a Fellow' of the Royal Society. Unfortunately, in 1917 Ramanujan became ex¬ 
tremely ill. At the time, it was thought that he had trouble with the English climate and had contracted 
tuberculosis. It is now thought that he suffered from a vitamin deficiency, brought on by Ramanujan’s strict 
vegetarianism and shortages in wartime England. He returned to rndia in 1919, continuing to do mathe- 
maties even when confined to his bed. He was religious and thought his mathematical talent came from 
his family deity, Namagiri. He considered mathematics and religion to be linked. He said that “an equation 
for me has no meaning unless it expresses a thought of God.” His short life came to an end in April 1920, 
when he was 32 years old. Ramanujan left several notebooks of unpublished results. The writings in these 
notebooks illustrate Ramanujan's insights hat are quite sketchy. Several mathematicians have devoted many 
years of study to explaining and justifying the results in these notebooks 
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EXAMPLE 2 


EXAMPLE 3 


algorithm I A Recursive Algorithm for Computing 

procedure poweria: nonzero real number, n: nonnegative integer) 

if n = 0 then poweria, n): = I 

else poweria, n) : = a * poweria, n - 1) 


Next we give a recursive algorithm for finding greatest common divisors. 


Give a recursive algorithm for computing the greatest common divisor of two nonneg¬ 
ative integers a and h with a < b. 

Solution: We can base a recursive algorithm on the reduction gcdi'a, b) = 
gedfh mod a, a) and the condition gcdfO, b) = b when b > 0. This produces the 
procedure in Algorithm 2. ■ 


algorithm 1 A Recursive Algorithm for Computing gedtu, b). 

procedure gedfa, b: nonnegative integers with a < b) 
if a = 0 then gedf a. b ): = b 
else gedfa, b ): = gcdf.h mod a, a) 


We will now give recursive versions of searching algorithms. 


Express the linear search algorithm as a recursive procedure. 

Solution: To search for x in the search sequence a\, a±, .. „ a n , at the fth step of the 
algorithm x and a, are compared. If x equals a, then i is the location of a. Otherwise, 
the search for x is reduced to a search in a sequence with one fewer element, namely, 
the sequence u,, i,_ a n . We can now give a recursive procedure. 

Let searchli, j, jt) be the procedure that searches for v in the sequence a,. a, r \ . a r 

The input to the procedure consists of the triple (1, n, x). The procedure terminates at a 
step if the first term of the remaining sequence is v or if there is only one term of the 
sequence and this is not x If a is not the first term and there are additional terms, the 
same procedure is carried out but with a search sequence of one fewer term, obtained 
by deleting the first term of the search sequence. ■ 


algorithm i A Recursive Sequential Search Algorithm. 

procedure search(i. j, x) 
if a, = jt then 
location : = i 

else if i = j then 

location 0 

else 

searchli +- 1, j, a) 
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EXAMPLE 4 


EXAMPLE 5 


Cuosirua a recursive version of a binary search algorithm. 

Solution: Suppose we warn to locate* in the sequence a \, m ,-. To perform a binary 
se arch, we begin by co mparing x with the middle term, n +., (/ 2( , Our algflri thm wi 11 term i- 
nale if v equals this term. Otherwise, we reduce the search to a smaller search sequence, 
namely, the first half of the sequence if x is smaller than the middle term of the original 
sequence, and the second half otherwise. We have reduced the solution of the search prob¬ 
lem to the solution of the same problem with a sequence approximately half as long. We 
express this recursive version of a binary search algorithm as Algorithm 4. ■ 


Al.i'rORrftrv 4 A Recursive Binary Search Algorithm. 

' procedure binary xearch(x, i. j) 

\ m : - |(r + j)ll\ 

; if x ~ ti m then 

location : -- m 

else if 1 v < a„ , and i < m ) then 
binary smrchix, i. tn — 1) 
else if fir > a,„ and j > m) then 
, binary searchix, m - l./l 

else htcatitm : -- 0 


RECURSION AND ITERATION 


A recursive definition expresses the value of a function ar a positive integer in terms of 
the values of the function at smaller integers, This means that we can devise a recursive 
algorithm to evaluate a recursively defined function at a positive integer. 


The following recursive procedure gives the value of n\ when the input is a positive 
integer n. M 


i UXiOKiTHM 5 A Recursive Procedure Cor Factorials 

I procedure faaonai(n: positive integer) 

j if n ^ 1 then 

jacloritdin) :- 1 

else 

j factonaiUi) := n *furt 0 mitin - 1) 


There is another way to evaluate the factorial function at art integer from its recur¬ 
sive definition. Instead of successively reducing the computation to the evaluation of 
the function at smaller integers, we can start with the value of the function at 1 and suc¬ 
cessively apply the recursive definition to find the values of the function a! successive 
larger integers. Such a procedure is called iterative. In other words, to find using an 
iterative procedure, we start with 1. the value of the factorial function at 1, and multiply 
successively by each positive integer less than or equal to n. This procedure is shown 
m Algorithm 6. 
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ALGORITHM ft Aft Iterative Procedure for Factorials. 

procedure iterative factorial!n: positive integer) 
a : = I 

for i := 1 to n 

x — i * \ 

{.v is w!} 


After this code has been executed, the value of the variable a is »!. For instance, going 
through the loop six times gives 6! = I ■ 2 ■ 3 • 4 • 3 ■ 6 - 720, 

Often an iterative approach for the evaluation of a recursively defined sequence re¬ 
quires much less computation than a procedure using recursion (unless special-purpose 
recursive machines arc used). This is illustrated by the iterative and recursive proce¬ 
dures for finding the nth Fibonacci number. The recursive procedure is given first. 


ALGORITHM 7 A Recursive Algorithm for Fibonacci Numbers. 

I 

procedure Jibmairi{n: nonnegalive integer) j 

| if n = 0 then fibonacci{0): — 0 
I else if a - 1 then fihanacci{ I): = I 

e\sefihon(icci(n) : = fibonacadi - 1) + ftbonacain - 2) 


When vve use a recursive procedure to find f n , we first express f„ as f n _ L -t- 2 . Then 

we replace both of these Fibonacci numbers by the sum of two previous Fibonacci 
numbers, and so on. When j\ or fy arises, it is replaced by its value. 

Note that at each stage of the recursion, until f or /q is obtained, the number of 
Fibonacci numbers to be evaluated has doubled. For instance, when we find / 4 using 
this recursive algorithm, we must carry out all the computations illustrated in the tree 
diagram in Figure 1. This tree consists of a root labeled with /*, and branches from 
the root In vertices labeled with the two Fibonacci numbers J] and fi that occur in the 
reduction of the computation of // Each subsequent reduction produces two branches 
in the tree. This branching ends when fy and f\ are reached. The reader can verify that 
this algorithm requires f„ +l - I additions to find /„. 



fit 

A 


>7: 




* • 

/i ft 


Wo 


Wo 


FIT I RK i Evaluating/) Recursively. 
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Now consider the amount of computation required to find /„ using the following 
iterative approach. 


ALGORITHM s An Iterative Algorithm for Computing Fibonacci 
Numbers. 


procedure iterative Jibmm.ci(n: nonnegativc integer) 

if n - 0 then y: - 0 

else 

begin 

x:=* 0 
v := 1 

for i 1 to « - 1 
begin 

Z'.= x 4- v 
x : = v 

end 

end 

{y is the nth Fibonacci number} 



This procedure initializes x as = 0 and y as /[ = 1. When the loop is traversed, the 
sum of.r and y is assigned to the auxiliary variable z. Then x is assigned the value of y 
and y is assigned the value of the auxiliary variable z- Therefore, after going through the 
loop the first time, it follows that x equals f] and y equals /o + f\ = /j. Furthermore, 
after going through the loop n - 1 times, x equals f n \ and y equals f„ (the reader should 
verify this statement). Only » - 1 additions have been used to find f n with this iterative 
approach when n > 1. Consequently, this algorithm requires far less computation than 
does the recursive algorithm. 

We have shown that a recursive algorithm may require far more computation than 
an iterative one when a recursively defined function is evaluated. It is sometimes prefer¬ 
able to use a recursive procedure even if it is less efficient than the iterative procedure, 
In particular, this is true when the recursive approach is easily implemented and the it¬ 
erative approach is not. (Also, machines designed to handle recursion may be available 
that eliminate the advantage of using iteration.) 


Exercises 


1. Give a recursive algorithm for computing nx whenever 
n is a positive integer and .v is an integer. 

2. Give a recursive algorithm for finding the sum of the 
first n positive integers. 

3. Give a recursive algorithm for finding the sum of the 
first ii odd positive integers 

4. Give a recursive algorithm For finding the maximum of 
a finite set of integers. 

5. Give a recursive algorithm for finding (he minimum of 
a finite set of integers. 


h. Devise a recursive algorithm for finding x* modm 
whenever n, a. and m are positive integers. 

7. Give a recursive algorithm for finding n! mod m when¬ 
ever it and m are positive integers. 

8. Give a recursive algorithm for finding a mode of a list 
of integers, (A mode is an element in the list that occurs 
at least as often as every other element. ) 

9. Devise a recursive algorithm for computing the great¬ 
est common divisor of two non negative integer 
n and b with a < b if gcdftt, b) = ged [a,b - a). 
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10. Devise a recursive algorithm to find a'" where a is a 
real number and n is a positive integer [Hint: Use die 
equality' <r" 1 ~ (<t 2 ") 2 ,] 

11. How does the number of multiplications used by the al¬ 
gorithm in Exercise 10 compare to the number of mul¬ 
tiplications used by Algorithm 1 to evaluate a 1 " 7 

*12. Use the algorithm in Exercise 10 to devise an algo¬ 
rithm for evaluating a n when n is a nonnegative integer. 
(Hint: Use the binary expansion of n.) 

* 13. How does the number of multiplications used by the al¬ 
gorithm in Exercise 12 compare to the number of mul¬ 
tiplications used by Algorithm 1 to evaluate a"? 

14. How- many additions are used hy the recursive and iter¬ 
ative algorithms given in Algorithms 7 and 8, respec¬ 
tively, to find the Fibonacci number /j? 

15. Devise a recursive algorithm to find the nth term of the 
sequence defined by «o = 1, uj = 2, and a„ = a„-i ■ 
a n -zi for n - 2, 3, 4,.... 

16. Devise an iterative algorithm to find the nth term of the 
sequence defined in Exercise 15. 

17. Is the recursive algorithm or the iterative algorithm for 
finding the sequence in Exercise 15 more efficient? 

18. Devise a recursive algorithm to find the nth term of 


the sequence defined by no -- I, ih = 2, ai - 3, and 
= a„ i + a n -2 + a n - } , for n = 3.4,5..... 

19. Devise an iterative algorithm to lind die nth term of the 
sequence defined in Exercise 18. 

20. Is the recursive algorithm or the iterative algorithm for 
finding the sequence in Exercise 18 more efficient? 

21. Give iterative and recursive algorithms for finding the 
nth term of the sequence defined by = 1, m = 3, 

= 5, and a„ = a n [ ■ - rri _ ? . Which is more 

efficient? 

22. Give a recursive algorithm to find the number of parti¬ 
tions of a positive integer based on the recursive defi¬ 
nition given in Exercise 35 in Section 3.3. 

23. Give a recursive algorithm for finding the reversal 
of a bit string, (Sec the definition of the reversal of 
a bit string in the preamble of Exercise 26 in Sec¬ 
tion 3.3.) 

24. Give a recursive algorithm for finding the string w 1 . the 
concatenation of i copies of w, when w is a bit string. 

25. Give a recursive algorithm for computing values of the 
Ackermann function, (Hint: See the preamble to Exer¬ 
cise 36 in Section 3.3.) 


15 ____ 

Program Correctness 

INTRODUCTION 

Suppose that we have designed an algorithm to solve a problem and have written a pro¬ 
gram to implement it. How can we be sure that the program always produces the correct 
answer? After all the bugs have been removed so that the syntax is correct, we can test 
the program with sample input. It i s not correct if an incorrect result is produced for any 
sample input. But even if the program gives the correct answer for all sample input, it 
may not always produce the correct answer (unless all possible input has been tested). 
We need a proof to show that the program always gives the correct output. 

Program verification, the proof of correctness of programs, uses the rules of infer¬ 
ence and proof techniques described in this chapter, including mathematical induction. 
Since an incorrect program can lead to disastrous effects, a large amount of methodology 
has been constructed for verifying programs. Efforts have been devoted to automating 
program verification so that it can be carried out using a computer. However, only lim¬ 
ited progress has been made toward this goal. Indeed, some mathematicians and theo¬ 
retical computer scientists argue that it will never be realistic to mechanize the proof of 
correctness of complex programs. 

Some of the concepts and methods used to prove that programs are correct will be 
introduced in this section. However, a complete methodology for program verification 
will not be developed in this book. This section is meant to be a brief introduction to the 
area ot program verification, which tics together the rules of logic, proof techniques, 
and the concept of an algorithm. 
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PROGRAM VERIFICATION 


A program is said to be correct if it produces the correct output for ever)' possible input. 
A proof that a program is correct consists of two parts. The first part shows that the 
correct answer is obtained if the program terminates. This part of the proof establishes 
the partial correctness of the program. The second pan of the proof shows that the 
program always terminates. 

To specify what it means for a program to produce the correct output, two propo¬ 
sitions are used. The first is the initial assertion, which gives the properties that the 
input values must have. The second is the final assertion, which gives the properties 
that the output of the program should have, if the program did what was intended. The 
appropriate initial and final assertions must be provided when a program is cheeked. 

DEFINITION 1. A program, or program segment, S is said to be partially correct 
with respect to the initial assertion p and die final assertion q If 1 whenever pi* true for 
theinputvainesof5'aitdi5tenninates,tiKai^isfiii£fortfieou^utyaiuos of5'. The 
notation p{S)q indicates that the program, or program segment, Sis partially correct 
with respect to the initial assertion p and the final assertion g, Note: The notation 
pfiS}^ is known as a Hoars triple after Tony Hoare, who introduced the concept of 
partial correctness. 


Note that the notion of partial correctness has nothing to do with whether a program 
terminates; it focuses only on whether the program does what it is expected to do if it 
terminates. 

A simple example illustrates the concepts of initial and final assertions. 

EXAMPLE I Show that the program segment 

y 2 
Z : =■- x + y 

is correct with respect to the initial assertion p: x = 1 and the final assertion q: z = 3. 

Solution: Suppose that/? is true, so that x = las the program begins. Then vis assigned 
the value 2, and z is assigned the sum of the values of x andy, which is 3. Hence, $ is cor¬ 
rect with respect to the initial assertion p and the final assertion q. Thus, p{S}q is true. ■ 


RULES OF INFERENCE 

A useful rule of inference proves that a program is correct by splitting the program into 
a scries of subprograms and then showing that each subprogram is correct. 


C, Anthony R. Hoare (born 19341. Tony Hoare is currently Professor of Computer Science at Oxford 
W€b University. England, and is a Fellow of the Royal Society. Hoare has made many important contributions to 
the theory of programming languages and to programming methodology. He was the first person to define 
a programming language based on how programs could he proved to be correct with respect to their spec¬ 
ifications. Hoare is also the creator of the quick sort, one of the most commonly used and studied sorting 
algorithm? (see the exercise set in Section 8.4J. Hoare is a noted writer in the technical and social aspects 
of computer science. 
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EXAMPLE 2 


Suppose that the program S is split into subprograms A| and .A. Write S = S \; S 2 
to indicate that S is made up of Si followed by Sn. Suppose that the correctness of Sj 
with respect to the initial assertion p and final assertion q, and the correctness of S 2 
with respect to the initial assertion q and the final assertion r, have been established. 
It follows that if p is true and Si is executed and terminates, then q is true; and if q is 
true, and S 2 executes and terminates, then r is true. Thus, if p is true and S = S\ \$2 is 
executed and terminates, then r is true. This rule of inference, called the composition 
rule, can be stated as 

p{S\}q 

q{Hr 

:.p{SuS 2 \r. 

This rule of inference will be used later in this section, 

Next, some rules of inference for program segments involving conditional state¬ 
ments and loops will be given. Since programs can be split into segments for proofs of 
correctness, this will let us verify many different programs. 


CONDITIONAL STATEMENTS 


First, rules of inference for conditional statements will be given. Suppose that a program 
segment has the form 


if condition then 
S 


where S is a block of statements. Then S is executed if condition is true, and it is not 
executed when condition is false. To verify that this segment is correct with respect to 
the initial assertion p and final assertion q, two things must be done. First, it must be 
shown that when p is true and condition is also true, then q is true after S terminates. 
Second, It must be shown that when p is true and condition is false, then q is true (since 
in this case S does not execute). 

This leads to the following rule of inference; 

(p A condirion ){S}q 
[p A -i condition ) —* q 

p{if condition then S)q. 

The following example illustrates how this rule of inference is used. 


Verify that the program segment 


if r > y then 

J: = * 


is correct with respect to the initial assertion T and the final assertion v > x. 
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EXAMPI.E 3 


Solution: When the initial assertion is true and x > y, the assignment y := x is carried 
out. Hence, the final assertion, which asserts that y - x, is true in this case. Moreover, 
when the initial assertion is true and x > y is false, so that x y, the final assertion 
is again true. Hence, using the rule of inference for program segments of this type, this 
program is correct with respect to the given initial and final assertions. ■ 


Similarly, suppose that a program has a statement of the form 


if condition then 
Si 

else 

Si 


Jf condition is true, then S\ executes; if condition is false, then 3; executes. To verify 
that this program segment is correct with respect to the initial assertion p and the final 
assertion 17, two things must be done. First, it must be shown that when p is true and 
condition is true, then q is true after terminates. Second, it must be shown that when 
p is true and condition is false, then q is true after Si terminates. This leads to the 
following rule of inference: 

ip A condtiion){S[}q 
(p A ^ccrndition'iiSitq 
.*. /){if condition then S ; else 

The following example illustrates how this rule of inference is used. 


Verify that tile program segment 


if x < 0 then 

abs := -x 

else 

abs : — x 


is correct with respect to the initial assertion T and the final assertion abs - jx|. 

Solution: Two things must be demonstrated. First, it must be shown that if the ini¬ 
tial assertion is true and x < 0, then abs = |.tj. This is correct, since when x < 0 the 
assignment statement abs:= -x sets a A: = - ,v, which is \x\ by definition whenx < 0. 
Second, it must be shown that if the initial assertion is true and x < 0 is false, so that 
x & 0, then abs - |jc|. This is also correct, since in this case the program uses the 
assignment statement abs : - x, and x is |x| by definition when x > 0, so that abs : - x. 
Hence, using the rule of inference for program segments of this type, this segment is 
correct with respect to the given initial and final assertions. ■ 
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EXAMPLE 4 


LOOP INVARIANTS 


Next, proofs of correctness of while loops will be described. To develop a rule of infer- 
HW ence for program segments of the type 


while condition 
S 


note that S is repeatedly executed until condition becomes false. An assertion that re¬ 
mains true each time S is executed must be chosen. Such an assertion is called a loop 
invariant. In other words,/) is a loop invariant if (p A condition){S}p is true. 

Suppose that p is a loop invariant, It follows that if p is true before the program 
segment is executed,/) and -i condition are true after termination, if it occurs. This rule 
of inference is 

(p A condninni{S)p 

p{while condition S}( -i condition A p). 

The use of a loop invariant is illustrated in the following example. 


A loop invariant is needed to verify that the program segment 


i: = 1 

factorial: - 1 

while i < n 
begin 

i: = i + 1 

factorial : = factorial * t 

end 


terminates with factorial - n\ when n is a positive integer. Let p be the proposition 
“factorial : = ;! and i £ n.” We will prove that p is a loop invariant using mathemat¬ 
ical induction. First, note that p is true before the loop is entered, since at that point 
factorial =1-1! and 1 - n. Now assume that p is true and i < n after an execu¬ 
tion of the loop. Assume that the while loop is executed again. First, i is incremented 
by 1. Thus, i is still less than or equal to n, since by the inductive hypothesis i < n 
before the loop was entered, and i and n are positive integers. Furthermore, facto¬ 
rial, which was (i - 1)! by the inductive hypothesis, is set equal to (i - 1)! • i = i!. 
Hence, p remains true. Therefore p is a loop invariant. In other words, the assertion 
[p A(i < ji)]{S}/j is true. It follows that the assertion /jfwhile i < n 5}[(i > n) A /;] is 
also true. 

Furthermore, the loop terminates after n - 1 traversals with i ~ n, since i is as¬ 
signed the value 1 at the beginning of the program, 1 is added to i at each traversal, and 
the loop terminates when i n, Consequently, at termination factorial = n\. ■ 
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A final example will be given to show how the various rules of inference can be 
used to verify the correctness of a longer program. 


EXAMPLE 5 We will outline how to verify the correctness of the program S for computing the product 

of two integers. 


procedure multiply{m, n: integers) 

if n < 0 then a := -n 

e!se a : - n 

k := 0 
x := 0 

while k < a 
begin 

x:~ x + m 
k: = k + 1 

end 


Si< 


S 4 


if n < 0 then product ; = -x 
else product: = x 


The goal is to prove that after. 5 s is executed, product has the value mn, The proof of correct¬ 
ness can be carried out by splitting S into four segments, with S = 5j; 2b; S 3 ; £ 4 , as shown 
in the listing of S'. The rule of composition can be used to build the correctness proof. Here 
is how the argument proceeds. The details will be left as an exercise for the reader. 

Let p be the initial assertion that m and n are integers. Then, it can be shown that 
p{S [ }q is true, when q is the proposition p A (a = |nj). Next, let r be the proposition 
q A (k = 0) A (,v = 0 ). It is easily verified that is true. It can be shown that 
"x = mk .irid k •" <j” is an invariant for the loop in S 3 . Furthermore, it is easy to see 
that the loop terminates after a iterations, with k = a, so that r - ma at this point. 
Since r implies that x = m • 0 and 0 •- a, the loop invariant is true before the loop 
is entered. Since the loop terminates with k - a , it follows that r{Si\s is true where 
v is the proposition u x = ma and a = fnj,” Finally, it can be shown that £4 is correct 
with respect to the initial assertion ,? and final assertion l, where t is the proposition 
product = mn" 

Putting all this together, since p{5\ }q, et-jlA }r, r{Sj)r, and ,f{5 4 }; are all true, it fol¬ 
lows from the rule of composition that ’pS}t is true. Furthermore, since ail four segments 
terminate, S does terminate. This verifies the correctness of the program. ■ 


Exercises 

1, Prove that the program segment 

y ; ■■ I 

c; = .1 + y 

is correct with respect to the initial assertion x = 0 and 
the final assertion ;; - 1. 


2, Verify that the program segment 
if x < 0 then x : = 0 

is correct with respect to the initial assertion T and the 
final assertion x > 0. 
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3. Verify that the program segment 

,t; = 2 
z A + V 

if v > 0 then 

z : = ; + 1 

else 

0 

is correct with respect to the initial assertion y -= 3 and 
the final assertion z = 6. 

4. Verify that the program segment 

if j < v then 

min : = x 

dse 

mm : = y 

is correct with respect to the initial assertion T and 
the final assertion U • y A min = x)\/ (.* > y A 
min - y). 

*5. Devise a rule of inference for verification of partial cor¬ 
rectness nf statements of the form 

if condition I then 

Si 

else if condition 2 then 
A] 

eke 

S„ 

where A, A 1 ;_5, are blocks. 

6. Use the rule of inference developed in Exercise 5 to 
verify that the program 

if x < 0 then 

y : = - 2UI/.V 

else if r > 0 then 

>‘ : - 2[^-r 

else if ,r ^ 0 then 

>■: = 2 

is correct with respect to the initial assertion T and the 
final assertion y = 2. 


7. Use a loop invariant to prove that the following pro¬ 
gram segment for computing the nth power, where n is 
a positive integer, of a real number x is correct, 

power : =■ I 
i r» 1 

while i £ n 
begin 

power : = power * x 

i : = i + 1 

end 

*8. Prove that the iterative program for finding f„ given in 
Section 3,4 is correct, 

9. Provide all the details in the proof of correctness given 
in Example 5. 

10. Suppose that both the implication pa — 1 ■ p t and the 
program assertion p]{S)q are true. Show that Pn^S\q 
also must be true. 

11. Suppose that both the program assertion p\S\q a and the 
implication q a —> r/j are true. Show that /r{.5ji?i also 
must be true, 

12. The following program computes quotients and re¬ 
mainders. 

r: = u 
( i '■ ~ 0 

while r a- <1 
begin 

r:= r - d 

?:= <i + 1 

end 

Verify that it is partially correct w ith respect to the ini¬ 
tial assertion “a and d are positive integers” and the fi¬ 
nal assertion “q and r are integers such that a - dq + r 
and 0 s r < d " 

13. Use a loop invariant to verify that the Euclidean algo¬ 
rithm (Algorithm 1 in Section 2,4) is partially correct 
with respect to the initial assertion “a and b are positive 
integers” and the final assertion “jr = gcd(u, b)? 


Key Terms and Results 


TERMS 

theorem: a mathematical assertion that can be shown to be 
true 

conjecture: a mathematical assertion whose truth value is 
unknown 

proof: a demonstration that a theorem is true 
lemma: a simple theorem used to prove other theorems 


corollary: a proposition that can be proved as a conse¬ 
quence of a theorem that has just been proved 
rule of inference: an implication that is a tautology which 
is theft used to draw conclusions from known asser¬ 
tions 

fallacy: an implication that is a contingency that is often 
incorrectly used to draw conclusions 
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circular reasoning or begging the question: reasoning 
where one or more steps are based on the truth of the 
statement being proved 

vacuous proof: a proof that the implication p — * q is true 
based on the fact that p is false 
trivial proof: a proof that the implication p q is true 
based on the fact that q is true 
direct proof; a proof that the implication p —* q is true 
that proceeds by showing that q must be true when p is 
true 

indirect proof: a proof that the implication p —> q is true 
that proceeds by showing that p must be false when q is 
false 

proof by contradiction: a proof that a proposition p is true 
based on the truth of the implication -ip -*■ q where q is 
a contradiction 

proof by cases: a proof of an implication where the hy¬ 
pothesis is a disjunction of propositions that shows that 
each hypothesis separately implies the conclusion 
counterexample; an element a - such that P(x") is false 
mathematical induction: a proof technique for statements 
of the form Vrt £ N P(n) consisting of a basis step and an 
inductive step 

basis step: the proof of P[ 1) in a proof of Vu 6 N Pitt) by 
mathematical induction 

inductive step: the proof of P(n) —* Pin + 1) in a proof 
ofVrr £ NT (it) by mathematical induction 
recursive definition of a function: a definition of a func¬ 
tion that specifies an initial set of values and a rule for 


obtaining values of this function at integers from its val¬ 
ues at smaller integers 

recursive definition of a set: a definition of a set th at spec¬ 
ifies an initial set of elements in the set and a rule for ob¬ 
taining other elements from those in the set 
recursive algorithm: ati algorithm that proceeds by reduc¬ 
ing a problem to the same problem with smaller input 
iteration; a procedure based on the repeated use of opera¬ 
tions in a loup 

program correctness: verification that a procedure al¬ 
ways produces the correct result 
loop invariant: a property that remains true during every 
traversal of a loop 

initial assertion: the statement specifying the properties 
of the input values of a program 
final assertion: the statement specifying the properties 
the output values should have if the program worked 
correctly 

RKSt I.I'S 

The well-ordering property: Every nonempty set of non¬ 
negative integers has a least element. 

Principle of mathematical induction: The statement 
Vrt P(n ) is true if P{ I ) is true and Vn[P(n) —» Pin + I)] 
is true. 

Second principle of mathematical induction: The state¬ 
ment Vn Pin) is true if P(l) is true and Vn[(P(l}A A 
FOlB — 1 2 3 4 5 Pin t- J)] is true, 


Review Questions 


1. a) Describe what is meant by a direct proof, an indirect 

proof, and a proof by contradiction of an i mplication 
P T 

b) Give a direct proof, an indirect proof, and a proof 
by contradiction of the statement: “If n is even, then 
n 4- 4 is even." 

2. a) Describe one way to prove the biconditional p q, 
b) Prove the statement: "The integer in + 2 is odd if 

and only if the i nigger ‘in + 5 is even, where n is an 
integer “ 

3. To prove that the statements pi, /?;, p 3 , and p 4 arc 
equivalent, is it sufficient to show that the implica¬ 
tions pi . P:, -■ p i, and pt — p 2 arc valid? 
If mil, provide another set of implications thal can 
be used to show that the four statements are equiva¬ 
lent. 

4. ai Suppose that a statement of the form Vx Pi.O is 

false. How cati this be proved? 
b) Show that the statement “For every positive integer 
«. the number n' ■■ I is prime" is false. 

5. a) What is the difference between a constructive 


existence proof and a nonconstructive existence 
proof? 

b) Show that for every integer n there is an integer 
greater than n that is not divisible by 3 or 5, Is your 
existence proof constructive or nonconstructive? 

6. a) Stare the well-ordering property for the set of posi¬ 

tive integers, 

b) Use this property to show that every' positive integer 
can be written as the product of primes, 

7. a) Can you use the principle of mathematical induction 

to find a formula for the sum of the first n terms of 
a sequence? 

b) Can you use the principle of mathematical induc¬ 
tion to determine whether a given formula for the 
sum of the first n terms of a sequence is correct? 

c) Find a formula forthe sum of the first n even positive 
integers, and prove it using mathematical induction. 

8. a) For which positive integers n is it true that lift 

17 2-"? 

b) Prove the conjecture you made in part (a) using 
mathematical induction. 
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9. aj Which amounts of postage can be formed using only 
5-C£nl and 9-cent stamps? 

b) Prove the conjecture you made using mathematical 
induction. 

c) Prove the conjecture you made using the second 
principle of mathematical induction. 

d) Find a proof of your conjecture different from the 
ones you gave in (b) and (c), 

10. G ive three different examples of proofs that use the sec¬ 
ond principle of mathematical induction. 

11. a) Explain why a function is well-defined if it is de¬ 

fined recursively by specifying /(l) and a rule for 
finding /(n) from fin ~ 1). 
h) Provide a recursive definition of the function 
fit 0 =(«+!)!. 

12. aj Give a recursive definition of the Fibonacci num¬ 

bers. 

b) Show that /„ > a" -1 whenever n £ 3 where /„ is 
the nth term of the Fibonacci sequence and a = 
(l t v 

13. aj Explain why a sequence «„ is well-defined if it is 

defined recursively by specifying «] and ai and a 
rule for finding a„ from ai, fli,..., u rt _i for n — 
3,4,5. 

b) Find the value of a„ if«[ = 1, a 2 = 2, and a„ = 
a„- 1 + n.-j + ••• + fj[. for n - 3,4, 5,_ 


14. Give two examples of how well-formed formulae are 
defined recursively for different sets of elements and 
operators. 

15, a) Give a recursive definition of the length of a string, 
b) Use the recursive definition from part (a) to prove 

that l(xy) = i(x) + Hy). 

IS, a) What is a recursive algorithm? 

b) Describe a recursive algorithm for computing the 
sum of n numbers in a sequence. 

17. Describe a recursive algorithm for computing the great¬ 
est common divisor of two positive integers. 

18. a) Does testing a computer program to sec whether it 

produces the correct output for certain input values 
verify that the program always produces the correct 
output? 

b) Docs showing that a computer program is partially 
correct with respect to an initial assertion and a final 
assertion verify that the program always produces 
the correct output ? If not, what else is needed ? 

19. What techniques can you use to show that a long com¬ 
puter program is partially correct with respect, to an ini¬ 
tial assertion and a final assertion? 

20. What is a loop invariant? How is a loop invariant 
used? 


Supplementary Exercises 

1. Prove that the product of two odd numbers is odd. 

2. Prove that v5 is irrational. 

3. Prove or disprove that the sum of two irrational num¬ 
bers is irrational. 

4. Prove or disprove that tf -t- n + 1 is prime whenever n 
is a positive integer. 

5. Determine whether the following is a valid argument. If 
n is greater than 5. then n 2 is greater than 25, Therefore, 
if n is an integer with n 1 greater than 25, it follows that 
n is greater than 5. 

6. Prove that it 4 — 1 is divisible by 5 when n is not divisible 
by 5. Use a proof by cases, with four different cases— 
one for each of the nonzero remainders that an integer 
not divisible by 5 can have when you divide it by 5. 

7. Prove that |.vy| = ,xj |>j by cases. 

*8. We define the Ulam numbers by setting in = 1 and 
“i = 2. Furthermore, after determining w hether the in¬ 
tegers less than n arc Ulam numbers, we set n equal 
to the next Ulam number if it can be written uniquely 
as the sum of two different Ulam numbers. Note that 
= 3. «4 = 4, «j = 6, and = 8. 

a) Find the first 20 Ulam numbers. 

b) Prove that there are infinitely many Ulam numbers. 


9, Give a constructive proof that there is a polynomial 
P(x) such that Fix,) = y,, P(x z ) = y 2 ,..., PU „) = 

_y„, where jq,..., jr*, vj. y„ arc real numbers. 

[Hint: Let 




v,] 


10. Show that I : ^3 5 6 7 +5 3 + --' + (2/1+1)* = in ■ FrC.-r • 
4?! + 1) whenever n is a positive integer. 

11. Show that 1 ■ 2° + 2 ■ 2' + 3 ■ 2 2 + ■ ■ ■ + n ■ 2" 1 =r 
(n - I) - 2" + 1 whenever n is a positive integer, 

12. Show that 

-L + _L +...+ __*_ 

1-3 3-5 (2/i - I)(2 m + 1) 2n+l 

whenever n is a positive integer. 

13. Show that 

-L + _!_+...+__ 1 _ = 

1-4 4-7 (3n - 2)(3n + 1) 3/1+1 

whenever n is a positive integer. 

14. Use mathematical induction to show that 2" > tf + n 
whenever n is an integer greater than 4. 
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JS. Lise mulbeiiiiiticai induction to show that 2" > n* 
■whenever n is an integer greater than 9. 

16. Hind an integer such that 2" > n 4 whenever n is 
greater than N. Prove that your result is correct using 
mathematical induction. 

17. Use mathematical induction to prove that a — b is a 
factor ota" - b" 'whenever n is a positive integer. 

18. Use mathematical induction to prove that 9 divides 
« 3 t in a l) 3 + in + 2) 1 whenever n is a nonnegative 
integer. 

19. An arithmetic progression is a sequence of the form 
a, ti + d, a 4 2d ,,.,, a + nd where a and d are real num¬ 
bers, Use mathematical induction to prove that the sum 
of these terms of an arithmetic progression is given by 

a 4- (a - d) + ■■■ + (a - 1 - nd) = in + l,ila 4- nd)/2. 

20. Suppose that ci; = b, (modml for j = 1,2 . n. Use 

mathematical induction to prove that 

a) a, = ^ h, (mod m) 

i»i 

b) | | «j = | ] ft; (mod mi 
t-i " i--i 

*21, Determine which Fibonacci numbers arc even, and 
use a form of mathematical induction to prove your 
conjecture. 

*22, Determine which Fibonacci numbers are divisible by 
3. Use a farm of mathematical induction to prove your 
conjecture. 

*23. Prove that/(/„ 4 = fmm for all non nega¬ 

tive integers n, where It is a nonnegativc integer and J] 
denotes the ith Fibonacci number 

The sequence of Lucas numbers is defined by 1$ = 2, i = 

1 . and !„ = i„. j + l n . 2 for n = 2, 3,4. 

24. Show that f„ 4 /n.i = /„+] whenever n is a positive 
integer, where f and L are the ith Fibonacci number 
and ith Lutati number, respectively. 

25. Show that + /j - ■ + C~ = + 2 whenever n 

is a nonnegative integer and /, is the ith Lucas number. 

*26. Use mathematical induction to show' that the product of 
any /i consecutive positive integers is divisible by rtf, 
[Him: Use the identity m(m + ]) ■ ■ ■ (m + n - l)/«! - 
!m - 1 )m(m f 1)--■ (m + n~2)In'. +m(.m+ 1) ■ ■■ (m + 
n - 2V(n - U'-l 

27. Use mathematical induction to show that (cos* 4 
f sin x)” = cos nx + ; sinru whenever n is a posi¬ 
tive integer, [Hint: Use the identities cos in 4 f>) =■ 
cos it cos b - sin a sin b and sitt(c 4 b) = sinacosb 4 
cos a sin h,\ 

*28. Use mathematical induction to show that T” t cos jx 
- cos/l/r t tl.rf2]sirifrEx/2}5in(.f/2) whenever n is a 
positive integer and sin(*/2j 0. 

The McCarthy 91 function is defined using the rule 


| n- 1° if n > 100 

n ~~ { M(M(n 4 11)) ifn <100 

for all positive integers n, 

29, By successively using the defining rule for Min), find 

a) Af(102), b)M(UM). c) M(99). 

d) 4/(97). e) AT(87). f) Af(76). 

**30, Show that the function iVf(ri) is a well-defined function 
front the set of positive integers to the set of positive 
integers. [Hint: Prove that Min) = 91 for all positive 
integers n with n s I0I.I 
31. Is the following proof that 

J_ _J_ + _1_ 3 \_ 

l -2 + 2-3 + f (h- 1)r ~ 2 "" n’ 

whenever n is a positive integer, correct? Justify your 
answer. 

BASIS STEP: The result is true when n = ! since 
J = 3 _ 1 

i ■ 2 ~ 2 r 

INDUCTIVE STEP: Assume that the result is true for 
n, Then 

L _L _ 1 1 

1-2 2-3 (n - l)n ntn 4 1) 



= 3 - 1 
2 « + I’ 

Hence, the result is irue for n + 1 if it is true for ri. This 
completes the proof. 

*32, A jigsaw puzzle is put together by successively join¬ 
ing pieces that tit together into blocks. A move is made 
each time a piece is added to a block, or w hen two 
blocks are joined. Use the second form of mathemat¬ 
ical induction to prove that no matter how the moves 
are earned out, exactly n - 1 moves are required to as¬ 
semble a puzzle with n pieces, 

*33. Show that n circles divide the plane into ir - n 4 2 re¬ 
gions if every two circles intersect in exactly two points 
and no three circles contain a common point. 

*34, Show' that n pi anes divide three-dimensional space into 
(n ? 4 5t: t 6}/6 regions if any three of these planes have 
a point in common and no four contain a common point. 

*35, Use the well-ordering property to show' that J2 is ir¬ 
rational. (Hint: Assume that y f 2 is rational. Show that 
the set of positive integers of the form h v /2 has a least 
element a. Then show' that ajl -a is a smaller positive 
integer of this form.) 

36. A set is w ell-ordered if every nonempty subset of this 
set has a least element. Determine whether each of the 
following sets is well-ordered, 
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a) the set of integers 

h) the set of integers greater than -100 
e) the set of positive rationale 
d) the set of positive rationals with denominator less 
than 100 

*37. Show that the well-ordering property can be proved 
when the principle of mathematical induction is taken 
as an axiom. 

*38. Show that the first and second principles of mathemati¬ 
cal induction are equivalent; that i s, each can he shown 
to be valid from the other. 

39. a) Show that if ai. 03 ,..., a„ are positive integers, 

then gcdffl;, ui, - -., a B .. ;. a „) = gcd(cq, aj . 

a„ gcdCi-j,, 1 . tiff )). 

b) Use part (a), together with the Euclidean algorithm, 
to develop a recursive algorithm for computing the 
greatest common divisor of a set of n positive inte¬ 
gers. 

*40, Describe a recursive algorithm for writing the greatest 
common divisor of« positive integers as a linear com¬ 
bination of these integers. 

41. Find art explicit formula for fin) if /(l) = 1 and 
fin) = f[n —1)4 2/7 — 1 fur ej ; 2 2. Prove your result 
using mathematical induction. 

**42. Ciivc a recursive definition of the set of bit strings that 
contain twice as many Os a.s I s. 

43. 1 ,et S be the set of hit strings defined recursively by 
A e 5 and Ojc E S', x 1 £ S if x E S. where A is the 
empty string. 

a) Find all strings in S of length not exceeding five. 

b) Give on explicit description of the elements of S. 

44. Let S be the set of strings defined recursively by 
abt E 5, bac E S. act E S. and ahex E S; ab.xc E S. 
ax be E S', xabc € S if ,t£S, 

a) Find all elements of S of length eight or less, 

b) Show that every clement of 5 has a length divisible 
by three 


The set B of all balanced strings of parentheses is defined 

recursively by A € B, where A is the empty string; (x) E fl, 

.tv t= B if x. y E B. 

45. Find all balanced strings of parentheses with four or 
fewer symbols. 

46. Use induction to show that if x is a balanced string of 
parentheses, then the number of left parentheses equals 
the number of right parentheses in x. 

Define the function M on the set of strings of parentheses by 
Nik) = 0,37(0 = I, /V( I) = -1, 

N(uv) = Niu ) + N(v). 

where A is the empty string, and u and v arc strings. It can 

be shown that N is well-defined. 

47. Find 

a) A'(O) b) Af( ))XM) 

O -VI (t )(: '.) d) N({ )((()))(())) 

**48, Show that a string w of parentheses is balanced if and 
only if ,V(w) = 1) and Niu) • 0 whenever u is a prefix 
of h that is, w - uv, 

*49. Give a recursive algorithm for finding all balanced 
strings of parentheses containing n or fewer symbols. 

50. Give a recursive algorithm for finding the greatest 
common divisor of two nonnegative integers a and b 
with a b, based on the fact that got! a, b) = a if 
a = b, ged(a, b) = 2 gcd{e/2, hi 2) if a and h are even, 
gedfu, b) s= (a/2. b) if a is even and b is odd, and 
gedia b) - gcd(h - a, b) if a and b are odd. 

51. Verify the program segment 

if x > y then 

x : = y 

with respect to the initial assertion T and the final as¬ 
sertion x - y. 

*52. Develop a rule of inference for verifying recursive pro¬ 
grams and use it to verify the recursive program for 
computing factorials given in Section 3.4. 


Computer Projects 

WRITE PROGRAMS WITH Tllf FOLLOWING INPUT AND OUTPUT. 


1, Given a geometric progression a. ar.ar 2 . ar ", find 

the sum of its terms. 

2. Given a nonnegative integer n. find the sum of the n 
smallest positive integers. 

**3. Given a 2" X 2* chessboard with one square missing, 
construct a tiling of this chessboard using L-shaped 
pieces. 

**4. Generate all well-formed formulae for expressions in¬ 
volving the variables x, y, and ; and the operators 
{-v, *. i, \ with u or fewer symbols. 

**5. Generate all well-formed formulae for propositions with 


n or fewer symbols where each symbol is T. F, one of 
the propositional variables p and q, or an operator from 

K V. A - *-*)■ 

6. Given a string, find its reversal. 

7. Given a real number a and a nonnegative integer n, find 
a' 1 using recursion, 

8. Given a real number a and a nonnegative integer n, find 
cr using recursion, 

*9. Given a real number a and a nonnegative integer n. find 
it' 1 using the binary- expansion of « and a recursive al¬ 
gorithm for computing rri*. 
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10. Given two integers not both zero, find their greatest 
common divisor using recursion. 

11. Given a list of integers arid an element x, locate x in 
this list using a recursive implementation of a linear 
search. 

12. Given a list of integers and an dement x, locate x in 
this list using a recursive implementation of a binary 
search. 

13. Given a nonnegative integer n, find the nth Fibonacci 
number using iteration- 


14, Given a nonnegative integer n, find the nth Fibonacci 
number using recursion. 

15, Given a positive integer, find the number of partitions 
of this integer. (See Exercise 35 of Section 3.3.) 

Iff. Given positive integers m and n, find ,4 (m, n), the value 
of Aekermann’s function at the pair (m, n). (See the 
preamble to Exercise 36 of Section 3.3.) 


Computations and Explorations 

USE A COMPUTATIONAL PROGRAM OR PROGRAMS YOU HAVE WRITTEN TO DO THE FOLLOWING EXERCISES. 


1. Verify Goldbach’s conjecture, which states that every 
even positive integer n is the sum of two primes, for 
,1 ■■ 10 , 000 . 

2. Find the smallest prime factor of nl + L for all positive 
integers n with n < 20. 

3. Find the smallest set of n consecutive composite inte¬ 
gers for each positive integer n with n 10, 

4. An old unsettled conjecture states that there are in¬ 
finitely many twin primes, that is. primes that differ by 
two. How many twin primes can you find? 

5. Determine which Fibonacci numbers are divisible by 
5, which are divisible by 7, and which are divisible by 
11. Prove that your conjectures are correct. 

fi. Construct tilings using L-shaped pieces of various 16x 
16,32 x 32, and 64 x 64 chessboards with one square 
missing, 


7. Explore which m X n chessboards can be completely 
covered by L-shaped pieces. Can you make a conjec¬ 
ture that answers this question? 

8. The notorious 3 x + 1 conjecture (also known as the 
Collalz conjecture and by many other names) states 
that no matter which integer x you start with, iterating 
the function fix), where fix) = x/2 if x is even and 
/( x) - 3 t + 1 if x is odd, always produces the integer 

1. Verify this conjecture for as many positive integers 
as possible, 

9. Which values of Ackerman it's function are small 
enough that you are able to compute them? 

10. Compare either the number of operations or the time 
needed to compute Fibonacci numbers recursively ver¬ 
sus that needed to compute them iteratively. 


Writing Projects 

RESPOND TO THE FOLLOWING WITH ESSAYS USING OUTSIDE SOURCES. 


1. Describe the origins of mathematical induction. Who 
were the first people to use it and to which problems 
did they apply it? 

2. In the past 20 years, several important theorems have 
been proved based on extensive computer computa¬ 
tions, Discuss the validity of such proofs and describe 
the controversy surrounding proofs based on computer 
calculations. 

3. Logic programming operates on statements expressed 
using quantifiers, predicates, and logical connectives 
using titles of inference. Explain the fundamental con¬ 
cepts of logic programming and how it is used in 
artificial intelligence. Illustrate its use with the pro¬ 
gramming language PROLOG, 


4. “Automate d theore m prOv ing” i s the task of using com - 
puters to mechanically proof theorems. Discuss the 
goals and applications of automated theorem proving 
and the progress made in developing automated theo¬ 
rem pro vers. 

5. Describe the basic rules of WFF’N PROOF, The Game 
of Modem Logic, developed by Layman Allen. (Be¬ 
cause this was published in the mid-1960s, you may 
have to have a scavenger hunt to find a set.) Give exam¬ 
ples of some of the games included in WFF’N PROOF. 

6. The L-shaped pieces used in the exercises of Section 
3.2 are examples of polyominos, introduced by Golomb 
lit 1954. Describe some of the problems and associated 
results concerning tiling chessboards with polyominos. 
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7. Discuss the uses of Ackcrm arm's function both in the 
theory of recursive definitions and in the analysis of the 
complexity of algorithms for set unions. 

8 . Describe some of the logical problems found in the 
writings of Lewis Carroll and show how rales of in¬ 
ference are used to solve these problems. 


9. Discuss some of the various methodologies used to es¬ 
tablish the correctness of programs and compare them 
to Hoare s methods described in Section 3.5. 

10. Explain how the ideas and concepts of program correct¬ 
ness can be extended to prove that operating systems 
are secure. 
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Counting 


C ombinatorics, the study of arrangements of objects, is an important part of dis¬ 
crete mathematics. This subject was studied as long ago as the seventeenth 
century, when combinatorial questions arose in the study of gambling games. 
Enumeration, the counting of objects with certain properties, is an important part of 
combinatorics. We must count objects to solve many different types of problems. For 
instance, counting is used to determine the complexity of algorithms. Counting is also 
required to determine whether there are enough telephone numbers or Internet proto¬ 
col addresses to meet demand. Furthermore, counting techniques are used extensively 
when probabilities of events are computed. 

The basic rules of counting, which we will study in Section 4 1, can solve a tremen¬ 
dous variety of problems. For instance, we can use these rules to enumerate the different 
phone numbers possible in the United States, the allowable passwords on a computer 
system, and the different orders in which the runners in a race can finish. Another im¬ 
portant combinatorial tool is the pigeonhole principle, which we will study in Section 
4.2. This states that when objects are placed in boxes and there are more objects than 
boxes, then there is a box containing at least two objects. For instance, we can use this 
principle to show that among a set of 15 or more students, at least 3 were bom on the 
same day of the week. 

We can phrase many counting problems in terms of ordered or unordered arrange¬ 
ments of the objects of a set. These arrangements, called permutations and combina¬ 
tions, are used in many counting problems. For instance, suppose the 100 tup finishers 
on a competitive exam taken by 2000 students arc invited to a banquet, We can enu¬ 
merate the possible sets of 100 students that will be invited, as well as the ways the top 
10 prizes can be awarded. 

We can analyze gambling games, such as poker, using counting techniques. We 
can also use these techniques to determine the probabilities of winning lotteries, such 
as the probability a person will win a lottery where 6 numbers are chosen from the first 
411 positive integers. 

Another problem in combinatorics involves generating all the arrangements of a 
specified kind. This is often important in computer simulations, We will devise algo¬ 
rithms to generate arrangements of various types. 


4,1 __ 

The Basies of Counting 

INTRODUCTION 


.A password on a computer system consists nf six, seven, or eight characters. Each of 
these characters must be a digit or a letter of the alphabet. Each password must contain at 
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EXAMPLE 1 


EXAMPLE 2 


least one digit. How many such passwords are there? The techniques needed to answer 
this question and a wide variety of other counting problems will be introduced in this 
section. 

Counting problems arise throughout mathematics and computer science. For ex¬ 
ample, we must count the successful outcomes of experiments and all the possible out¬ 
comes of these experiments to determine probabilities of discrete events. We need to 
count the number of operations used by an algorithm to study its time complexity. 

We will introduce the basic techniques of counting in this section. These methods 
serve as the foundation for almost all counting techniques. 


BASIC COUNTING PRINCIPLES 


We will presen t two basic counting principles. Then we will show how they can be used 
to solve many different counting problems. 

THE SUM RULE If a first task can be done in ways and a second task in n 2 
ways, and if these tasks cannot be done at the same time, then there are rq +■ n 2 ways 
to do either task. 


The following example illustrates how the sum rule is used. 


Suppose- that either a member of the mathematics faculty or a student who is a mathe¬ 
matics major is chosen as a representative to a university committee. How many differ¬ 
ent choices are there for this representative if there are 37 members of the mathematics 
faculty and 83 mathematics majors? 

Solution: The first task, choosing a member of the mathematics faculty, can be done 
in 37 ways. The second task, choosing a mathematics major, can be done in 83 ways. 
From the sum rule it follows that there are 37 + 83 = 120 possible ways to pick this 
representative. ■ 


We can extend the sum rule to more than two tasks. Suppose that the tasks 
7j, T 2 ,.. -, T m can be done in n i, ttj, ..., n m ways, respectively, and no two of these 
tasks can be done at the same time. Then the number of w'ays to do one of these tasks 
is m + »2 4 ■ ■ ■ + n.„. This extended version of the sum rule is often useful in counting 
problems, as Examples 2 and 3 show 1 . This version of the sum rule can be proved using 
mathematical induction from the sunt rule for two sets. (This is Exercise 53 at the end 
of the section. ) 


A student can choose a computer project from one of three lists. The three lists contain 
23,15, and 19 possible projects, respectively. How many possible projects are there to 
choose from? 

Solution: The student can choose a project from the first list in 23 ways, from the second 
list in 15 w r ays, and from the third list in 19 ways. Hence, there are 23 + 15 + 19 = 57 
projects to choose from. ■ 
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EXAMPLE 3 


EXAMPLE 4 


What is the value of k after the following code has been executed? 


k := 0 

for /, : = 1 to n \ 
k £ + 1 
for h 1 to ri 2 

k.^k+l 


for i m := 1 to n m 
k: = k+ 1 

f 

Solution: The initial value of k is zero. This block of code is made up of m different 
loops. Each time a loop is traversed, 1 is added to k, Let T, be the task of traversing the 
ith loop. The task T, can be done in n, ways, since the ith loop is traversed n, times. 
Since no two of these tasks can be done at the same time, the sum rule show's that the 
final value of k, which is the number of ways to do one of the tasks 7j, i = 1,2, ..,,rn, 
is Mi 4 m + ■ ■ ■ + tt m . ■ 


The sum rule can be phrased in terms of sets as follows: If A\, A;__ A m are 

disjoint sets, then the number of elements in the union of these sets is the sum of the 
numbers of elements in them. To relate this to our statement of the sum rale, let 7", be the 
task of choosing an element from A, for i - 1,2,..., m. There are |A,j ways to do T,. 
From the sum rule, since no two of the tasks can be done at the same time, the number 
of ways to choose an element from one of the sets, which is the number of elements in 
the union, is 

|A] U At U 1 ' 1 U A m | — (A11 + [ 4- ■ • ■ + |A m j. 

This equality applies only when the sets in question are disjoint. 'The situation is much 
more complicated when these sets have elements in common. That situation will be 
briefly discussed later in this section and discussed in more depth in Chapter 5. 

The product mle applies when a procedure is made up of separate tasks. 

THE PRODUCT RULE Suppose that a procedure can be broken down Into two 
tasks. If there are m ways to do the first task and ways to do tfce second task after 
the first task has been done, then there are m rn ways to do the procedure. 

The following examples show hew the product rule is used. 


The chairs of an auditorium are to be labeled with a letter and a positive integer not 
exceeding 100. What is the largest number of chairs that can be labeled differently? 

Solution: The procedure of labeling a chair consists of two tasks, namely, assigning 
one of the 26 letters and then assigning one of the 100 possible integers to the seat. 
The product rule shows that there are 26 ■ 100 - 2600 different ways that a chair 
can be labeled. Therefore, the largest number of chairs that can be labeled differently 
is 2600. ■ 
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EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


EXAMPLE 9 


There are 32 microcomputers in a computer center. Each microcomputer has 24 ports. 
How many different ports to a microcomputer in the center are there? 

Solution: The procedure of choosing a port consists of two tasks, first picking a micro¬ 
computer and then picking a port on this microcomputer. Since there are 32 ways to 
choose the microcomputer and 24 ways to choose the port no matter which microcom¬ 
puter has been selected, the product rule shows that there are 768 ports. ■ 


An extended version of the product rule is often useful. Suppose that a procedure 
is carried out by performing the tasks T h 7a,..., T m . If task T, can be done in n, ways 

after tasks 7), Tj ,..., and T,-_] have been done, then there are n i - m.ways to 

carry out the procedure. This version of the product rule can be proved by mathematical 
induction from the product rule for two tasks (see Exercise 54 at the end of the section). 


How many different bit strings are there of length seven? 

Solution: Each of the seven bits can be chosen in two ways, since each bit is either 
zero or one. Therefore, the product rule shows there are a total oil 1 = 128 different 
bit strings of length seven. ■ 


How many different license plates are available if each plate contains a sequence of 
three letters followed by three digits (and no sequences of letters are prohibited, even 
if they are obscene)? 

.Solution: There are 26 choices for each of the three letters and 10 choices for each of 
the three digits. Hence, by the product rule there are a total of 26 ■ 26 ■ 26 ■ 10 ■ 10 ■ 10 = 
17,576,000 possible license plates. ■ 


Counting Functions How many functions are there from a set with rn elements to one 
with n elements? 

Solution: A function corresponds to a choice of one of the n elements in the codomain 
for each of the m elements in the domain. Hence, by the product rule there are n ■ n ■ 
■ ■ • ■ n = n m functions from a set with m dements to one with n elements, ■ 


Counting One-to-Onc Functions How' many one-to-one functions are there from a set 
with m elements to one with n elements? 

Solution: First note when m > n there are no one-to-one functions from a set with m 
elements to a set with n elements. Now' let m ^ n. Suppose the elements in the domain 

are aj. a m . There are n ways to choose the value of the function at uj. Since 

the function is one-to-one, the value of the function at a? can be picked in n - 1 ways 
(since the value used for a\ cannot he used again). In general, the value of the function 
at in can be chosen in n - k + 1 ways. By the product rule, there arc n(n - l)(n - 2) ■ ■ ■ 
(n - m + 1) one-to-one functions from a set with m elements to one with n elements. 
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The Telephone Numbering Plan The format of telephone numbers in North America 
is specified by a numbering plan, A telephone number consists of 10 digits, which 
are split into a three-digit area code, a three-digit office code, and a four-digit station 
code. Because of signaling considerations, there are certain restrictions on some of these 
digits. To specify the allowable format, let X denote a digit that can take any of the 
values 0 through 9, let N denote a digit that can take any of the values 2 through 9, and 
let Y denote a digit that must be a 0 or a 1. Two numbering plans, which will be called 
the old plan and the new plan, will be discussed, (The old plan, in use in the 1960s, has 
been replaced by the new' plan, but the recent rapid growth in demand for new numbers 
will make even this new plan obsolete.) As will be shown, the new plan allows the use 
of more numbers. 

In the old plan, the formats of the area code, office code, and station code are NYX. 
NNX, and XXXX, respectively. In the new' plan, the formats of these codes are /VAX 
XXX, and XXXX, respectively. How many different North American telephone numbers 
are possible under the old plan and under the new plan? 

Solution: By the product rule, there are 8 2 tO = 160 area codes with format ATX 
and 8*10-10 - 800 area codes with format/VXX. Similarly, by the product rule, there 
are 8 * 8 ■ 10 = 640 office codes with format NNX and 8 ■ 10 ■ 10 = 800 with format 
XXX. The product rule also shows that there are 10 -10 ■ 10 ■ 10 = 10,000 station codes 
with format XXXX, 

Consequently, applying the product rule again, it follows that under the old plan 
there are 

160 ■ 640 ■ 10,000 - 1,024,000,000 

different numbers available in North America. Under the new plan there are 
800 ■ 800 ■ 10,000 = 6,400,000,000 

different numbers available, ■ 


EXAMPLE U What is the value of k after the following code has been executed? 


EXAMPLE 10 


k 0 

for (i := 1 torii 
for h 1 to «-> 


for i m : - 1 to n„ 
k: = k + 1 


Solution: The initial value of k is zero. Each time the nested loop is traversed, 1 is 
added to k. Let T, be the task of traversing the ith loop. Then the number of times the 
loop is traversed is the number of ways to do the tasks T u T 2 ,.... T m . The number of 

ways to carry out the task 7j = 1,2. trt, is n,, since the/th loop is traversed once 

for each integer i } with 1 < i } tij. By the product rule, it follows that the nested 
loop is traversed nj ■ ■ - n m times. Hence, the final value of k is mnj ■ • ■ n m . ■ 
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EXAMPLE 12 


EXAMPLE 13 


EXAMPLE 14 


Counting Subsets of a Finite Set Use the product rule to show that the number of dif¬ 
ferent subsets of a finite set S is 2.I 5 . 

Solution: Let S be a finite set. List the elements of S in arbitrary order. Recall that 
there, is a one-to-one correspondence between subsets of S and bit strings of length |5J. 
Namely, a subset of 5 is associated with the bit string with a 1 in the j'th position if the 
>'tli element in the list is in the subset, and a 0 in this position otherwise, By the product 
rule, there are 2l 5 l bit strings of length |5j. Hence, jP(S)j = 2 is l ■ 


The product rule is often phrased in terms of sets in the following way: If 
Ai ,A% A m are finite sets, then the number of elements in the Cartesian prod¬ 
uct of these sets is the product of the number of elements in each set. To relate this 
to the product rule, note that the task of choosing an element in the Cartesian product 

Ai X A 2 X ■ ■ * X A m is done by choosing an element in A \, an element in A?.and 

an element in A m . From the product rule it follows that 

|Aj xAx-XAj = |A,| ■ \A 2 \ . \A m \. 

MORE COMPLEX COUNTING PROBLEMS Many counting problems cannot 
be solved using just the sum rule or just the product rule. However, many complicated 
counting problems can be solved using both of these rules. 


In a version of the computer language BASIC, the name of a variable is a string of 
one or two alphanumeric characters, where uppercase and lowercase letters are not 
distinguished, (An alphanumeric character is either one of the 26 English letters or 
one of the 10 digits.) Moreover, a variable name must begin with a letter and must be 
different from the five strings of two characters that are reserved for programming use. 
How many different variable names are there in this version of BASIC? 

Solution: Let V equal the number of different variable names in this version of BASIC. 
Let Vi be the number of these that are one character long and V 2 be the number of 
these that are two characters long. Then by the sum rule, V = Vy + V 2 . Note that 
V\ =26, since a one-character variable name must be a letter. Furthermore, by the 
product rule there are 26 - 36 strings of length two that begin with a letter and end 
with an alphanumeric character. However, five of these are excluded, so that V 2 = 
26-36 — 5 = 931. Hence, there are V = V, + V 2 = 26 + 931 = 957 different names 
for variables in this version of BASIC. ■ 


Each user on a computer system has a password, which is six to eight characters long, 
where each character is an uppercase letter or a digit. Each password must contain at 
least one digit. How- many possible passwords are there? 

Solution; Let P be the total number of possible passwords, and let F b , P 7 , and Pg denote 
the number of possible passwords of length 6, 7, and 8, respectively. By the sum rule, 
P = P 6 + Pi + P g . We will now find P 6 , P lt and P%, Finding directly is difficult. To 
find P h it is easier to find the number of strings of uppercase letters and digits that are 
six characters long, including those with no digits, and subtract front this the number 
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of strings with no digits. By the product rule, the number of strings of six characters is 
36 6 , and the number of strings with no digits is 26*. Hence, 

P, = 36 f ’ - 26 6 = 2,176,782,336 - 308,915,776 = 1,867,866,560. 

Similarly, it can be shown that 

Pi = 36 7 - 26 7 = 78,364,164,096 - 8,031,810.176 - 70,332,353,920 
and 

P% --- 36* - 26* = 2,821,109,907,456 - 208,827.064,576 
= 2,612282,842,880. 

Consequently, 

P = P t , + Pi + P % = 2.684,483,063,360. ■ 


EXAMPLE 15 Counting Internet Addresses. In the Internet, which is made up of interconnected 

physical networks of computers, each computer (or more precisely, each network con- 
■ r nection of a computer) is assigned an Internet address. In Version 4 of the Internet 
Protocol (IPv4), now in use, an address is a string of 32 bits. It begins with a network 
number (netid). The netid is followed by a host number (hostid), which identifies a 
computer as a member of a particular network. 

Three forms of addresses are used, with different numbers of bits used for netids 
and hostids. Class A addresses, used for the largest networks, consist of 0, followed 
by a 7-bit netid and a 24-bit hostid. Class B addresses., used for medium-sized net¬ 
works, consist of 10, followed by a 14-bit netid and a 16-bit hostid. Class C addresses, 
used for the smallest networks, consist of 110, followed by a 21-bit netid and an S-6it 
hostid. There are several restrictions on addresses because of special uses; 1111111 is 
not available as the netid of a Class A network, and the hostids consisting of all Os and 
all Is are not available for use in any network. A computer on the Internet has either a 
Class A, a Class B, or a Class C address. (Besides Class A. B, and C addresses, there 
are also Class D addresses, reserved for use in multicasting when multiple computers 
are addressed at a single time, consisting of 1110 followed by 28 bits, and Class E ad¬ 
dresses, reserved for future use, consisting of 11110 followed by 27 bits. Neither Class 
D nor Class E addresses are assigned as the IP address of a computer on the Internet.) 
Figure 1 illustrates IPv4 addressing. (Limitations on the number of Class A and Class 
B netids have made IPv4 addressing inadequate; IPv6, which will replace IPv4, uses 
128-bit addresses to solve this problem.) 

How many different IPv4 addresses are available for computers on the Internet? 


Bit Number 

01234 8 16 24 31 
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FIGURE 1 Internet Addresses (IPv4). 
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EXAMPLE 16 


Solution: Let .i be the number of available addresses for computers on the Internet, 
and let x^, x», and X( denote the number of Class A, Class B, and Class C addresses 
available, respectively. By the sum rule, x = x r \+ x s -r .i,-. 

To find xa, note that there are 2 7 — 1 = 127 Class A netids, recalling that the 
netid 1111111 is unavailable. For each netid, there are 2 24 - 2 — 16,777,214 hostids, 
recalling that the hostids consisting of all Os and all 1 s are unavailable. Consequently, 
x A ~ 127- 16,777,214 = 2,130,706,178. 

To find xs and x c . note that there are 2 14 - 16,384 Class B netids and 2 21 = 
2.097.152 Class C netids. For each Class B netid, there are 2 16 - 2 - 65,534 hostids, 
and for each Class C netid, there are 2 s - 2 = 254 hostids, recalling that in each net¬ 
work the hostids consisting of all Os and all Is are unavailable. Consequently, x.g - 
1,073,709,056 and x c = 532,676,608. 

We conclude that the total number of IPv4 addresses available is x - T x& T 
x c = 2,130,706,178 + 1,073,709,056 + 532,676,608 - 3,737,091,842. ■ 


THE INCLUSION-EXCLUSION PRINCIPLE 


When two tasks can be done at the same time, we cannot use the sum rule to count the 
number of ways to do one of the two tasks. Adding the number of ways to do each task 
leads to an overcount, since the ways to do both tasks sire counted twice. To correctly 
count the number of ways to do one of the two tasks, we add the number of ways to 
do each of the two tasks and then subtract the number of ways to do both tasks. This 
technique is called the principle of inclusion-exclusion. Example 16 illustrates how 
we can solve counting problems using this principle. 


How many bit strings of length eight either start with a 1 bit or end with the two 
bits 00? 

Solution: The first task, constructing a bit string of length eight beginning with a I bit. 
can be done in 2 7 = 128 ways. This follows by the product rule, since the first bit can 
be chosen in only one way and each of the other seven bits can be chosen in two ways 
The second task, constructing a bit string of length eight ending with the two bits 
00, can be done in 2 6 = 64 ways. This follows by the product rule, since each of the first 
six bits can be chosen in two ways and the last two bits can be chosen in only one way. 
Both tasks, constructing a hit string of length eight that begins with a 1 and ends 
with 00. can be done in 2 5 = 32 ways. This follows by the product rule, since Lhc 
first bit can be chosen in only one way, each of the second through the sixth bits can 
be chosen in two w ays, and the last two hits can be chosen in one way. Consequently, 
the number of bit strings of length eight that begin with a 1 and end with a 00, 
which equals the number of ways to do either the first task or the second task, equals 128 
+ 64 - 32 = 160. ■ 


We can phrase this counting principle in terms of sets. Let Ay and ,4 2 be sets. Let 
T| be the task of choosing an element from A j and T 2 the task of choosing an element 
from A- : . There are |Aj | ways to do T\ and LA 2 ! w'ays to do Tj. The number of ways to 
do either 7*1 or T? is she sum of the number of ways to do T\ and the number of ways to 
do 7V. minus the number of ways to do both T\ and 73, Since there, are |A t U A. 2 \ ways 
to do either T\ or 73 and ;Ai H A;! ways to do birth Ty and 73, we have 

|Ai U A 2 \ — |A11 + |Aj| — [A| fi AtL 
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EXAMPLE 1? 


EXAMPLE 18 


This is the formula given in Section 1.5 for the number of elements in the union of two 
sets. 

The principle of inelusion-exclusion can be generalized to find the number of ways 
to do one of n different tasks or, equivalently, to find the number of elements in the 
union of n sets, whenever n is a positive integer. We will study the inclusion-exclusion 
principle and some of its many applications in Chapter 5. 


TREE DIAGRAMS 


Counting problems can be sol ved using tree diagrams. A tree consists of a root, a num¬ 
ber of branches leaving the root, and possible additional branches leaving the endpoints 
of other branches. (We will study trees in detail in Chapter 8.) To use trees in counting, 
we use a branch to represent each possible choice. We represent the possible outcomes 
by the leaves, which are the endpoints of branches not having other branches starting at 
them. 


How many bit strings of length four do not have two consecutive 1 s'? 

Solution: The tree diagram in Figure 2 displays all bit strings of length four without 
two consecutive Is. We see that there are eight bit strings of length four without two 
consecutive Is. ■ 


A playoff between two teams consists of at most five games. The first team that wans 
three games wins the playoff. In how many different ways can the playoff occur? 

Solution: The tree diagram in Figure 3 displays all the ways the playoff can proceed, 
with the wanner of each game shown. We see that there are 20 different ways for the 
playoff to occur. g 


0 



FIGURE 2 Bit Strings of Length Four Without Consecutive Is. 
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Team l 



Game I Game 2 Game 3 Game 4- Game 5 

FIGURE 3 Best Three Games Out of Five Playoffs. 


PDF created with pdfFactory trial version www.pdffactorv.com 




242 4 / Counting 


Exercises 

1. There are IS mathematics majors and 325 computer 
science majors at a college, 

a) How many ways are there to pick two representa¬ 
tives, so that one is a mathematics major and the 
other is a computer science major? 

b) How many ways are there to pick one representa¬ 
tive who is eitlier a mathematics major or a com¬ 
puter science major? 

2. An office building contains 27 floors and has 37 offices 
on each floor. How many offices are in the building? 

3. A multiple-choice test contains 10 questions. There are 
four possible answers for each question, 

a) How many ways can a student answer the questions 
on the test if every question is answered? 

b) How many ways can a student answer the ques¬ 
tions on the test if the student can leave answers 
blank? 

4. A particular brand of shirt comes in 12 colors, has a 
male version and a female version, and comes in three 
sixes for each sett. How many different types of this 
shirt are made? 

5. There are six different airlines that fly from Mew York 
to Denver and seven that fly from Denver to ban Fran¬ 
cisco, How many different possibilities are there for 
a trip from New York to San Francisco via Denver, 
when an airline is picked ior (he flight to Denver and 
an airline is picked for the continuation flight to 
San Francisco? 

6. There are four major auto routes from Boston to Detroit 
and six from Detroit to Los Angeles. How many major 
auto mules are there from Boston to Los Angeles via 
Detroit? 

7. How many different three-letter initials can people 
have? 

8. How many different three-letter initials with none of 
the letters repeated can people have? 

H. How many different three-letter initials are there that 
begin with an A? 

II). How many bit strings are there of length eight? 

11. How many bit strings of length 10 begin and end with 

a I? 

12. How many bit strings are Lhere of length six or less'* 

13. How many bit strings with length not exceeding n. 
where n is a positive integer, consist entirely of Is? 

14. How many bit strings of length n, w here n is a positive 
integer, start and end with Is? 

15. How many strings are there of lowercase letters of 
length four or Jess!' 

16. How many strings are there of four lowercase'Tettcrs 
that have the letter i in them? 

17. How many strings of five ASCII characters contain the 
character (u tat sign) at least once? (Mifev There arc 
12k different ASCII characters,) 


18. How many positive integers less than 1000 are 

a) divisible by 7? 

b) divisible by 7 hut not by 11? 
cl divisible by both 7 and 11 ? 

d) divisible by either 7 or 11 ? 

e) divisible by exactly one of 7 and 11 ? 

f) divisible by neither 7 nor 11? 

g) have distinct digits? 

h) have distinct digits and are even? 

19. How many positi ve integers with exactly three decimal 
digits, that is, positive integers between 100 and 999 
inclusive, 

a) are divisible by 7 ? 

b) are odd? 

c) have the same three decimal digits? 

d) are not divisible by 4? 

e) are divisible by 3 or 4? 

f) are not divisible by either 3 or 4? 

g) are divisible by 3 but not by 4? 

h) are divisible by 3 and 4? 

20. How many positive integers with exactly four decimal 
digits, that is. positive integers between 1000 and 9999 
inclusive, 

a) are divisible by 9? 
h) are even? 

Cl have distinct digits? 

d) are not divisible by 3? 

e) are divisible by 5 or 7? 

f) are not divisible by either 5 or 7 ? 

g) are divisible by 5 but not by 7? 

h) are divisible by 5 and 7? 

21. How many strings of three decimal digits 

a) do not contain the same digit three times? 

b) begin with an odd digit? 

c) have exactly two digits that are 4s? 

22. How many strings of four decimal digits 

a) do not contain the same digit twice? 

b) end with an even digit? 

c) have exactly three digits that are 9s? 

23. A committee is formed containing cither the governor 
or one of the two senators of each of the 50 states. How 
many ways are there to form this committee? 

24. How many license plates can be made using either 
three digits followed by three letters or three letters fol¬ 
lowed by three digits? 

25. How many license plates can be made using either two 
letters followed by four digits or two digits followed by- 
four letters? 

26. How many license plates can be made using either 
three letters followed by three digits or four letters fol¬ 
lowed by two digits? 

27. How many license plates can be made using either two 
or three letters followed by cither two or three digits? 
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28. How many strings of eight English letters are there 

a) if letters can be repeated? 

b) if no letter can be repeated? 

c) that start with X, if letters can be repeated? 

d) that start with X, if no letter can be repeated? 

e) that start and end with X, if letters can be repeated? 

f) that start with the letters BO (in that order), if letters 
can be repeated? 

g) that start and end with the letters BO (in that order), 
if letters can be repeated? 

h) that start or end with the letters BO (in that order), 
if letters can be repeated? 

29. How many strings of eight English letters arc there 

a) that contain no vow'els, if letters can be repeated? 

b) that contain no vowels, if letters cannot be repeated? 

c) that start with a vowel, if letters can be repeated? 

d) that start with a vowel, if letters cannot be repeated? 

e) that contain at least one vowel, if letters can be 
repeated? 

f) that contain exactly one vowel, if letters can be 
repeated? 

g) that start with X ami contain at feast one vowel, if 
letters can be repeated? 

h) that start and end with X and contain at least one 
vowel, if letters can be repeated? 

30. How many different functions are there from a set 
with 10 elements to sets with the following numbers of 
elements? 

a) 2 b) 3 e) 4 d) 5 

31. How many one-to-one functions are there from a set 
with live elements to sets with the following number 
of elements ? 

a) 4 b) 5 e) 6 d) 7 

32. How many functions are there from the set {1,2. r,}, 

where n is a positive integer, to the set (0,1}? 

33. How many functions are there from the set {1,2,_«}, 

where n is a positive integer, to the set {0,1} 

a) that arc one-to-one? 
h) that assign 0 to both 1 and«? 
c) that assign 1 to exactly one of the positive integers 
less than n? 

.34. How many partial functions (see the exercises in Sec¬ 
tion ! .6) are there from a set with five elements to sets 
with tiie following number of elements? 
a) 1 b) 2 c) 5 d) 0 
35. How many partial functions (see the exercises in Sec- 
Lion 1.6) are there from a set with m elements to a set 
with n elements, where m and n are positive integers? 
3b, How many subsets of a set with 100 elements have 
more than one element? 

37. A palindrome is a string whose reversal is identical to 
the string. How many bit strings of length n are palin¬ 
dromes? 

38. In how many ways can a photographer at a wedding 
arrange 6 people in a row from a group of 10 people. 


where the bride and the groom are among dtese 10 
people, if 

a) the bride must be in the picture? 

b) both the bride and groom must be in the picture? 

c) exactly one of the bride and the groom is in the pic¬ 
ture? 

39, In how many ways can a photographer at a wedding 
arrange six people in a row, including the bride and 
groom, if 

a) the bride must be next to the groom* 

b) the bride is not next to the groom? 

c) the bride is positioned somewhere to the left of the 
groom? 

40, How many bit strings of length seven either begin with 
two 0s or end with three Is? 

41, How many bit strings of length 10 either begin with 
three 0s or end with two 0s? 

*42. How' many bit strings of length 10 contain either five 
consecutive Os or five consecutive Is? 

**43. How many bit strings of length eight contain either 
three consecutive Os or four consecutive I s? 

44. Every student in a discrete mathematics class is either 
a computer science or a mathematics major or is a joint 
major in these two subjects. Haw many students are in 
the class if there are 38 computer science majors (in¬ 
cluding joint majors), 23 mathematics majors (includ¬ 
ing joint majors), and 7 joint majors? 

45, How many positive integers not exceeding 100 are 
divisible either by 4 or by 6? 

4fi. The name of a variable in the C programming lan¬ 
guage is a string that can contain uppercase letters, low¬ 
ercase letters, digits, or underscores. Further, the first 
character in the string must be a letter, either upper¬ 
case or lowercase, or aa underscore. If the name of a 
variable is determined by its first eight characters, how 
many different variables can be named in C? (Note that 
the name of a variable may contain fewer than eight 
characters) 

47. Suppose that at some future time every telephone in the 
world is assigned a number that contains a country code 
1 to 3 digits long, that is, of the form A', AX or AAA, 
followed by a 10-digit telephone number of the form 
NXX-NXX-XXXX (as described in Example 10), How 
many different telephone numbers would be available 
worldwide under this numbering plan? 

48. Use a tree diagram to find the number of bit strings of 
length four with no three consecutive Os. 

49. How many ways are, there to arrange the letters a, b s c, 
and d such that a is not followed immediately by hi 

50. Use a tree diagram to find the number of ways that the 
World Series can occur, where the first team that wins 
four games out of seven wins the series. 

51. Use a tree diagram to determine the number of subsets 
of {3,7, 9. If, 24} with the property that the sum of the 
elements in the subset is less that) 28. 
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*52, Use the product rule to show that there are 2 : " different 
truth tables for propositions m n variables, 

53, Use mathematical induction to prove the sum rule for 
m tasks from the sum rule for two tasks. 

54. Use mathematical induction to prove the product rule 
for m tasks from the product rule for two tasks. 

55, How many diagonals does a convex polygon with n 
sides have'/ (A polygon is convex if every line segment 
connecting two points in the interior or boundary of the 
polygon lies entirely within this set.) 

56. Data are transmitted over the Internet in datagrams, 
which are structured blocks of bits. Each datagram 
contains header information organized into a maxi¬ 
mum of i4 different fields (specifying many things, 
including the source and destination addresses) and 
a data area that contains the actual data which are 
transmitted. One of the 14 header fields is the header 
length field (denoted by HLEN), which is specified 
by the protocol to be 4 bits long and which specifies 
the header length in terms of 32-bit blocks of bits. For 
example, if HLEN = 0110, the header is made up of 


six 32-bit blocks. Another of the 14 header fields is 
the 16-hit-long total length field (denoted by TOTAL 
LENGTH), which specifies the length in bits of the 
entire datagram, including both the header fields 
and the data area. The length of the data area is the 
total length of the datagram minus the length of the 
header. 

a) The largest possible value of TOTAL LENGTH 
(which is 16 bits long) determines the maximum 
total length in octeLs (blocks of 8 bits) of an Internet 
datagram. What is this value ) 

b) The largest possible value of HLEN (which is 4 bits 
long) determines the maximum total header length 
in 32-bit blocks. What is this value? What is the 
maximum total header length in octets? 

c) The minimum (and most common) header length 
is 20 octets, What is the maximum total length in 
octets of the data area of an Internet datagram ? 

d) How many different strings of octets in the data area 
can be transmitted if the header length is 20 octets 
and the total length is as long as possible? 


42 __ 

The Pigeonhole Principle 


INTRODUCTION 


Suppose that a flock of pigeons flies into a set of pigeonholes to roost. The pigeonhole 
web principle states that if there are more pigeons than pigeonholes, then there must be 
at least one pigeonhole with at least tw o pigeons in it (see Figure )). Of course, this 
principle applies to other objects besides pigeons and pigeonholes. 

THEOREM l THE PIGEONHOLE PRINCIPLE If k +1 or more objects are placed into k boxes, 

then there is at least oae box containing mo or more of the objects. 

Proof: Suppose that none of the k boxes contains more than one object. Then the toial 
number of objects would be at most k. This is a contradiction, since there are at least 
k + J objects. o 

The pigeonhole principle is also called the Dirichlet drawer principle, after the 
nineteenth-century Gentian mathematician Dirichlet, who often used this principle in 
his work. The following examples show how the pigeonhole principle is used. 


G. I-fjeune Dirichlet (1805-1859). G, Ujeune Diricblet was bom into a French family living near 
web Cologne, Germany, He studied at the University of Paris and held positions at the University of Breslau and 
the University of Berlin. In 1K55 he was chosen to succeed Gauss at the University of Gottingen. Dinehlet 
i s said to be the first person ro master Gauss’s. Disqaisitmiws Arithmetics, which appeared 20 years earlier. 
He is said to have kept a copy al his side even when he traveled. Dinehlet made many important discoveries 
111 number theory, including the theorem that there are infinitely man)' primes in arithmetical progressions 
"" '■ t> when « Mid b arc relatively prime. Ho proved the n = 5 exit of Fermat's Last Theorem, that (here 
are no nontrivial solutions in integers to ,r 5 * y’ ■=■ p. Dinehlet also made many contribution [t: MudySiS. 
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EXAMPLE 1 


Among any group of 367 people, there must be at least two w ith the same birthday, 
because there are only 366 possible birthdays. ■ 


EXAMPLE 2 In any group of 27 English words, there must be at least two that begin with the same 

letter, since there are 26 letters in the English alphabet. ■ 


EXAMPLE 3 How many students must he in a class to guarantee that at least two students receive 

the same score on the final exam, if the exam is graded on a scale from 0 to 100 points? 

Solution: There are 101 possible scores on the final. The pigeonhole principle shows 
that among any 102 students there must be at least 2 students w ith the same score, ■ 

THE GENERALIZED PIGEONHOLE PRINCIPLE 

The pigeonhole principle states that there must be at least two objects in the same box 
when there are more objects than boxes. However, even more can be said when the 
number of objects exceeds a multiple of the number of boxes. For instance, among any 
set of 21 decimal digits there must be 3 that arc the same. This follows because when 
21 objects are distributed into 10 boxes, one box must have more than 2 objects. 

THEOREM 2 THE GENERALIZED PIGEONHOLE PRINCIPLE If N objects are placed into 

k boxes, then there is at least one box containing at least ]NJk] objects. 

Proof: Suppose that none of the boxes contains more than \N/k] - 1 objects. Then, the 
total number of objects is at most 

K\N/k] - 1) < ftCi.V/A) + 1) - I) = N, 

where the inequality [;V7Aj < (N/k) + 1 has been used. This is a contradiction since 
there are a total of N objects. □ 

The following examples illustrate how the generalized pigeonhole principle is applied. 
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EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


Among 100 people there are at least | r 1 CIO/I 2] = 9 who were bom in the same month. 


Whal is the minimum number of students required in a discrete mathematics class to 
be sure that at least six will receive the same grade, if there are five possible grades, A, 
B. C, D, and F? 

Solution: The minimum number of students needed to guarantee that at least six stu¬ 
dents receive the same grade is the smallest integer N such that fjV/5] = 6, The smallest 
such integer is N = 5-5+1 = 26. Thus, 26 is the minimum number of students needed 
to be sure that at least 6 students will receive the same grade, ■ 


What is the least number of area codes needed to guarantee that the 25 million phones 
in a state have distinct 10-digit telephone numbers? (Assume that telephone numbers 
are of the form NXX -NXX-XXXX, where the first three digits form the area code, N 
represents a digit from 2 to 9 inclusive, and A' represents any digit.) 

Solution: There are % million different phone numbers of the form NXX-XXXX (as 
shown in Example 10 of Section 4.1). Hence, hy the generalized pigeonhole princi¬ 
ple, among 25 million telephones, at least f25,000,000/8,000,000] of them must have 
identical phone numbers. Hence, at least four area codes are required to ensure that all 
10-digit numbers are different. ■ 

SOME ELEGANT APPLICATIONS OF THE 
PIGEONHOLE PRINCIPLE 


In many interesting applications of the pigeonhole principle, the objects to be placed in 
boxes must be chosen in a clever way. A few such applications will be described here. 


During a month with 30 days a baseball team plays at least 1 game a day, but no more 
than 45 games. Show that there must be a period of some number of consecutive days 
during which the team must play exactly 14 games. 

Solution: Let a s be the number of games played on or before the /th day of the month. 
Then a\, ..., %) is an increasing sequence of distinct positive integers, with 1 £ 

(ij ^ 45. Moreover. o,\ + 14, a-y + 14.Ujo + 14 is also an increasing sequence of 

distinct positive integers, with 15 s. a, + 14 < 59. 

The 60 positive integers 0 |, a } ,.,,, a Vi . a t + 14, a> + 14.ti M + 14 are all less 

than or equal to 59. Hence, by the pigeonhole principle two of these integers are equal. 

Since the integers ay, j = 1,2 .30 are all distinct and the Integers aj + 14, j = 

1.2.30 are all distinct, there must be indices i and j with a, = + 14. This means 

that exactly 14 games were played from day ; + 1 to day t. ■ 


Show that among any n + 1 positive integers not exceeding 2 n there must be an integer 
that divides one of the other integers. 
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THEOREM 3 


EXAMPLE 9 


Solution: Write each of the n + 1 integers a n+ \ as a power of 2 times an odd 

integer. In. other words, let a s - 2 k: q, tor j = 1.2,_n +1. where kj is a nonnegative 

integer and q s is odd. The integers q\,qj .,... q n 4 i are ali odd positive integers less 
than 2 n. Since there are only n odd positive integers less than 2n, it follows from the 
pigeonhole principle that two of the integers q\.qi,, -q n + \ must be equal. Therefore, 
there are integers (' and j such that q, = qj, Let q be the common value of q : and q ; . 
Then, a, = 2 ki q and aj - 2 ki q, It follows that if k, < kj, then a, divides a while if 
k, > kj, then a> divides a;. ■ 


A clever application of the pigeonhole principle show's the existence of an increas¬ 
ing or a decreasing subsequence of a certain length in a sequence of distinct integers. 
Some definitions will he reviewed before this application is presented. Suppose that 

an «2 .a,v is a sequence of real numbers. A subsequence of this sequence is a se- 

quence of the form a^.a ^,,. where 1 < ii < i 2 < •--< i ffl £ N. Hence, a sub¬ 
sequence is a sequence obtained from the original sequence by including some of the 
terms of the original sequence in their original order, and perhaps not including other 
terms. A sequence is called strictly increasing if each term is larger than the one that 
precedes it, and it is called strictly decreasing if each term is smaller than the one that 
precedes it. 


Every sequence of n 2 + 1 distinct real numbers contains a subsequence of length 
it + 1 that is either strictly increasing or strictly decreasing. 

The following example will be given before the theorem is proved. 


The sequence 8, 11,9, 1, 4, 6, 12, 10, 5, 7 contains 10 terms. Note that 10 = '3 2 + 1. 
There are four increasing subsequences of length four, namely, 1,4, 6,12; 1,4,6. 7; 1, 
4, 6 , 10; and 1, 4,5, 7. There is also a decreasing subsequence of length four, namely, 
11,9,6,5. ■ 


The proof of the theorem will now be given. 

Proof: Let a\, a?, ■... a„ 2 +[ he a sequence of >r + 1 distinct real numbers. Associate 
an ordered pair with each term of the sequence, namely, associate (4, du to the term 
a*, where 4 is the length of the longest increasing subsequence starting at a*, and rfj- 
is the length of the longest decreasing subsequence starting at a k . 

Suppose that there are no increasing or decreasing subsequences of length n + 1. 
Then 4 and d k are both positive integers less than or equal to n, for k ~ 1,2,,... rr + 
1. Hence, by the product rule there are n 2 possible ordered pairs for (4, d k ). By the 
pigeonhole principle, two of these tr + 1 ordered pairs are equal. In other words, there 
exist terms a s and a„ with s < i such that f - i, and d s - d t . We will show that this 
is impossible, Because the terms of the sequence are distinct, either a s < a, or a 5 > a r . 
If a s < a t , then, since i s - i t , an increasing subsequence of length i t + 1 can be built 
starting at a s , by taking a, followed by an increasing subsequence of length i, beginning 
at a,. This is a contradiction. Similarly, if a, > a,, it can be shown that d s must be greater 
than d, , which is a contradiction. □ 
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The final example shows how the generalized pigeonhole principle can be applied 
to an important part of combinatorics called Ramsey theory, after the English mathe¬ 
matician F. P. Ramsey. In general, Ramsey theory deals with the distribution of subsets 
of elements of sets. 


EXAMPLE 10 Assume that in a group of six people, each pair of individuals consists of two friends or 

two enemies. Show that there are either three mutual friends or three mutual enemies 
in the group. 

Solution: Let A be one of the six people. Of the five other people in the group, there are 
either three or more who are friends of A, or three or more who are enemies of A, This 
follows from the generalized pigeonhole principle, since when five objects are divided 
into two sets, one of the sets has at least [5/21 = 3 elements. In the former case, suppose 
that B , C, and D are friends of A. If tiny two of these three individuals are friends, then 
these two and A form a group of three mutual friends. Otherwise, B, C, and D form a 
set of three mutual enemies. The proof in the latter case, when there are three or more 
enemies of A, proceeds in a similar manner. ■ 


Exercises 

1. Show that in any set of six classes there must be two 
that meet on the same day, assuming that no classes 
me held on weekends. 

2. Show that if there are 30 students in a class, then at 
least 2 have last names that begin with the same letter. 

3. A drawer contains a dozen brown socks and a dozen 
black socks, all unmatched. A man takes socks out at 
random in the dark. 

a) How many socks must he take out to be sure that he 
has at least two socks of die same color? 

b) How many socks must he take out to be sure that he 
has at least two black socks? 

4. A bowl contains 10 red balls and 10 blue balls. A 
woman selects balls at random 'without looking at them. 

a) How many halls must she select to be sure of having 
at least three balls of the same color? 

b) How many bal Is must she select to be sure of having 
at least three blue balls? 


5. Show that among any group of five (not necessarily 
consecutive) integers, there are two with the same re¬ 
mainder when divided by 4. 

6. Let d be a positive integer. Show that among any group 
of d +- 1 {not necessarily consecutive) integers there 
are two with exactly the same remainder when they are 
divided by d. 

7. Let n be a positive integer. Show that in any set of 
n consecutive integers there is exactly one divisible 
by n. 

8. Show that if / is a function from 5 to T where S and T 
are finite sets with |S| > |7j, then there are elements sy 
and st in. S such that f(sj ) = /(jj), or in other words, 
/ is not one-to-one. 

9. How many students, each of whom comes from one of 
the 50 states, must be enrolled in a university to guaran¬ 
tee that there are at least 100 who come from the same 
state? 


Frank Plumpton Ramsey <1903-1934)1. Frank Plumpton Ramsey, son of' the president of Magdalene 
W&fa College, Cambridge, was educated at Winchester and Trinity Colleges. After graduating in 1923. he was 
elected a fellow of King s College, Cambridge, where he spent the remainder of his life. Ramsey made 
important contributions to mathematical logic. What we now call Ramsey theory began with his clever 
combinatorial arguments, published in the paper “On a Problem of Formal Logic," Ramsey also made con¬ 
tributions to the mathematical theory of economics. He was noted as an excellent lecturer on the foundations 
of mathematics. His death at the age of 26 deprived the mathematical community and Cambridge University 
of a brilliant young scholar. 
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*10. Let Uf. Vj). ‘ = I. 2, 3,4.5, be a set of five distinct 
points with integer coordinates in the xy plane. Show 
that the midpoint of the line joining at least one pair of 
these points has integer coordinates. 

*11. Let [jtf. Vi,i = 1,2, 3,4,5, 6, 7,8,9, he a set of nine 
distinct points with integer coordinates in xyz space. 
Show that the midpoint of at least one pair of these 
points has integer coordinates. 

12. How many ordered pairs of integers (a, b) are needed to 
guarantee that there are two ordered pairs (ai, by ) and 
(ai. bz } such that a\ mod 5 = an mod 5 and b\ mod 
5 = bn mod 5? 

13. a) Show that if five integers are selected from the first 

eight positive integers, there must he a pair of these 
integers with a sum equal to 9. 
b) Is the conclusion in part (a) true if four integers are 
selected rather than five? 

14. a) Show that if seven integers are selected from the 

first 10 positive integers, there must be at least two 
pairs of these integers with the sum 11. 
b) Is the conclusion in part (a) true if six integers are 
selected rather than seven? 

15. A company stores products in a warehouse. Storage 
bins in this warehouse are specified by their aisle, loca¬ 
tion in the aisle, and shelf. There are 50 aisles, 85 hori¬ 
zontal locations in each aisle, and 5 shelves throughout 
the warehouse. What is the least number of products die 
company can have so that at least two products must be 
stored in the same bin? 

16. There arc 51 houses on a street. Each house has an ad¬ 
dress between I (XX) and 1099, inclusive. Show that at 
least two houses have addresses that are consecutive 
integers. 

*17. Let x be an irrational number. Show that the absolute 
value of the difference between jx and the nearest in¬ 
teger to jx is less than 1 in for some positive integer j 
not exceeding n. 

18. Find an increasing subsequence of maximal length and 
a decreasing subsequence of maximal length in the se¬ 
quence 22, 5, 7. 2. 23, 10.15, 21,3, 17. 

19. Construct a sequence of 16 positive integers that has no 
increasing or decreasing subsequence of 5 terms. 

20. Show that if there are 101 people of different heights 
standing in a line, it is possible to find 1! people in the 
order they are standing in the line with heights that are 
either increasing or decreasing. 

*21. Describe an algorithm in pseudocode lor producing the 
largest increasing or decreasing subsequence of a se¬ 
quence of distinct integers. 

22. Show, that in a group of five people (where any two peo¬ 
ple are either friends nr enemies), there are not neces¬ 
sarily three mutual friends or three mutual enemies. 

23. Show that in a group of 10 people ( where any 2 people 
arc either friends or enemies), there are either 3 mutual 


friends or 4 mutual enemies, and there are either 3 mu¬ 
tual enemies or 4 mutual friends. 

24. Use Exercise 23 to show that among any group of 20 
people (where any 2 people are either friends or en¬ 
emies), there are either 4 mutual friends or 4 mutual 
enemies, 

25. Show that there are at least four people in California 
(population: 25 million) with the same three initials 
who were bom on the same day of the year (but mu 
necessarily in the same year), 

26. Show that if there are 100,(XX),000 wage earners in 
the United States who earn less than 1,000,000 dollars, 
then there arc two who earned exactly lire same amount 
of money, to the penny, last year. 

27. There are 38 different lime periods during which 
classes at a university can be scheduled. If there are 
677 different classes, how many different rooms will 
be needed? 

28. A computer network consists of six computers. Each 
computer is directly connected to at least one of the 
other computers. Show' that there are at least tw r o com¬ 
puters in the network that are directly connected to the 
same number of other computers. 

29. A computer network consists of six computers. Each 
computer is directly connected to zero or more of the 
other computers. Show' that there are at least two com¬ 
puters in the network that are directly connected to the 
same number of other computers. 

*30. Prove that at a party where there are at least two people, 
there are two people who know the same number of 
other people there. 

31. An arm wrestler is the champion for a peruxl of 75 
hours. The arm wrestler had at least one match an hour, 
but no more than 125 total matches. Show that there 
is a period of consecutive hours during w'hich the arm 
wrestler had exactly 24 matches. 

*32. Is the statement in Exercise 31 true if 24 is replaced by 
a) 2? b) 23? c) 25? d) 30? 

33. Show that if/ is a function from S to T where $ and T 
are finite sets and m - [|S|47jl, then there are at least 
m elements of S that are mapped to the same value of 

T. That is, show that there are elements Ji, Sn, _.v m 

of S such that /(*,) = f(s 2 ) = ■ ■ ■ = f(s m ), 

34. Suppose that there are nine students in a discrete math¬ 
ematics class at a small college. 

a) Show that the class must have at least five male stu¬ 
dents or at least five female students. 

b) Show that the class must have at least three male 
students or at least seven female students. 

35. Suppose that every student in a discrete mathematics 
class of 25 students is a freshman, a sophomore, or a 
junior. 

a) Show that there are at least 9 freshman, at least 9 
sophomores, or at least 9 juniors in the class. 
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EXAMPLE 6 


THEOREM 2 


COROLLARY 1 


We see that C(4,2) = 6, since the 2-combinations of {a, b, c,dj are the six subsets 
{a, b}, {a, c}, {a. d}, {b, c}, [b, d}, and {c, d). ■ 


We can determine the number of r-eombinations of a set with n elements us¬ 
ing the formula for the number of r-permutations of a set. To do this, note that the 
r-permutations of a set can be obtained by first forming r-combinations and then or¬ 
dering the elements in these combinations. The proof of the following theorem, which 
gives the value of C(n, r), is based on this observation. 


The number of Mxnahiiiaiioins of a set with s elements, wiere n is 

and r is m integer with 0 s r rs n, equals , , J „ 


C(n,r) = 


r! (« — r)! 


Proof: The r-permutations of the set can be obtained by forming the C(n, r) 
r-combinations of the set, and then ordering the elements in each r-eombmation, which 
can be done in P(r, r) ways. Consequently, 

Pin, r) = C(n, r) ■ P(r, r). 

This implies that 

C( = = nU(n ~ r)f = n - 

T P{r,r) r'.Kr - r)! r!(/i-r)!' □ 

The following corollary is helpful in computing the number of r-combinations of a set. 


Let« and r be nonncgative integers with r ^ n. Hen Cin, r) = C(h, ti ~ r). 
Proof: From Theorem 2 it follows that 


C(n, r) 


n'. 

r\ in - rj! 


and 


C(n, n - r) = 


(n - r)! [n - (n — r)]! (n - r)! r!' 

Hence, C(n, r) — C(n, n - r), □ 

There is another common notation for the number of r-combinations from a set with 
n elements, namely. 


This number is also called a binomial coefficient The name binomial coefficient is 
used because these numbers occur as coefficients in the expansion of powers of binomial 
expressions such as (a + b) n . We will discuss the binomial theorem, which expresses a 
power of a binomial expression as a sum of terms involving binomial coefficients, later 
in this section. 
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until there are exactly n - r + 1 ways to choose the rth element. Consequently, by the 
product rale, there are 

n(n - 1 )(ri - 2) - ■ - {n ~ r +■ 1) 

/■-permutations of the set. □ 

From Theorem 1 it follows that 

P(n, r) = n(n - \){n - 2) (n - r + 1) = nlf(n - r)\ 

In particular, note that Pin, ft) = n\. We will illustrate (his result with some examples. 

EXAMPLE 2 

How many different ways are there to select 4 different players from 10 players on a 
team to play four tennis matches, where the matches are ordered? 

Solution: The answer is given by the number of 4-permutations of a set with 10 ele¬ 
ments. By Theorem 1, this is P(1CI, 4) = 10-9-8-7 = 5040. ■ 

EXAMPLE 3 

Suppose that there are eight runners in a race. The winner receives a gold medal, the 
second-place finisher receives a silver medal, and the third-place finisher receives a 
bronze medal. How many different ways are there to award these medals, if all possible 
Gti[comes of the race can occur? 

Solution: The number of different ways to award the medals is the number of 

3-permutations of a set with 8 elements. Hence, there are PCS. 3) = 8 • 7 ■ 6 = 336 
possible ways to award the medals. ■ 

EXAMPLE 4 

Suppose that a saleswoman has to visit eight different cities. She must begin her trip in 
a specilied city, but she can visit the other seven cities in any order she wishes. How 
many possible orders can the saleswoman use when visiting these cities',’ 

Solution: The number of possible paths between the cities is the number of permuta¬ 
tions ot seven elements, since the first city is determined, but the remaining seven can 
be ordered arbitrarily. Consequently, there are 7! = 7 - 6 ■ 5 ■ 4 ■ 3 ■ 2 ■ 1 = 5040 ways 
for the saleswoman to choose her tour. If, for instance, the saleswoman wishes to find 
the path between the cities with minimum distance, and she computes the total distance 
for each possible path, she must consider a total of 5040 paths! ■ 

COMBINATIONS 

web 

An r-enmbiilation of elements of a set is an unordered selection of r elements from the 
set. Thus, an /--combination is simply a subset of the set with r elements. 

EXAMPLE 5 

Let S be the set {1,2, 3,4}. Then {1, 3.4} is a 3-combination from 5. ■ 

The number of /--combinations of a set with « distinct elements is denoted by Ct/i r). 
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EXAMPLE 6 


THEOREM 2 


COROLLARY I 


We see that C(4,2) = 6, since the 2-combinations of {a, b, c.d) are the six subsets 
{a, b}, {a. c}, {a, d\ {b, c}, {b, d\, and {c, t/}. ■ 


We can determine the number of r-combinations of a set with ft dements us¬ 
ing the formula for the number of r-permutations of a set. To do this, note that the 
r-pcrmutations of a set can be obtained by first forming r-combinations and then or¬ 
dering the elements in these combinations. The proof of the following theorem, which 
gives the value of C(n, r), is based on this observation. 


The number of /--co mbinations of a set wife n 
and r is an integer wife 0 s r < n, equals 
n! 


C(n,r) = 


r}{n - r)f 


elements, where n is a positi ve integer 


Proof: The /-permutations of the set can be obtained by forming the C(n, r) 
/■-combinations of the set, and then ordering the elements in each r-combination, which 
can be done in P(r, r ) ways. Consequently, 

Pin, r) - C(n, r) ■ Pin ; r>. 

This implies that 


Pin. n _ n'iin - r)\ _ n 
P{r,r) r'i'ir r\l r\ (n - r)! ’ 


O 


The following corollary is helpful in computing the number of/ -combinations of a set, 


Let ft and r be nonnegative integers with r ^ n. Then C(n, r) - C(n, n - r). 

Proof: From Theorem 2 it follows that 
n\ 


C(n, r) = — - 

r! (/i - /’)! 


and 


C(n, n - r) ~ -— 

(n - r)l [n - (n - r)]! (n - r)! r! 

Hence, C(n, r) = C(n, n - r) □ 

There is another common notation for the number of r-combinations from a set with 
n dements, namely, 


This number is also called a binomial coefficient. The name binomial coefficient is 
used because these numbers occur as coefficients in the expansion of powers of binomial 
expressions such as ia + bf. We will discuss the binomial theorem, which expresses a 
power of a binomial expression as a sum of terms involving binomial coefficients, later 
in this section. 
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EXAMPLE 7 


EXAMPLE 8 


THEOREM 3 


How many ways are there to select 5 players from a 10-member tenths team to make a 
trip to a match at another school? 

Solution: The answer is given by the number of 5-combinations of a set with 10 ele¬ 
ments. By Theorem 2, the number of such combinations is 

cno,5)-i^ r 2 52. B 


How many ways are there to select a committee to develop a discrete mathematics 
course at a school if the committee is to consist of 3 faculty members from the mathe¬ 
matics department and 4 from the computer science department, if there are 9 faculty 
members of the mathematics department and 11 of the computer science department? 

Solution: By the product rule, the answer is the product of the number of 3-combina¬ 
tions of a set with 9 elements and the number of 4-combinations of a set with 11 ele¬ 
ments. By Theorem 2. the number of ways to select the committee is 

C( 9 . 3 ) ■ CUl, 4) = 3 |, ■ ~ = 84 ■ 330 = 27 , 720 . , 

BINOMIAL COEFFICIENTS 


Some of the more important properties of the binomial coefficients will be discussed 
here. The first property to be discussed is an important identity. 

PASCAL’S IDENTITY Let n and k be positive integers with n > it. Then 
Cin + 1 , k) = C(n, k - 1 ) + Cin, k\ 

Proof: Suppose that T is a set containing n + l elements. Let a be an element in T, 
and let S = T - {a}. Note that there are On + 1 ,k) subsets of T containing k dements. 
However, a subset of T with k elements either contains a together with k - 1 elements 
of S, or contains k elements of S and does not contain a. Since there are Cin, k - 1) 
subsets of C I dements of S, there are C(n, k - 1) subsets of k dements of T that 
contain a. And there are Cin. k) subsets of k dements of T that do not contain a, since 
there are On, k) subsets of k elements of S. Consequently, 

C(n + 1, k) = C{n, k- 1) + C(n, k), □ 

Remark: A combinatorial proof of Pascal *s identity has been given. It is also possible to 
prove this identity by algebraic manipulation from the formula for Cin, r) (see Exercise 
47 at the end of this section). 

Pascal’s identity is the basis for a geometric arrangement of the binomial coeffi¬ 
cients in a triangle, as shown in Figure 1. 

The nth row in the triangle consists of the binomial coefficients 

!)■ t_(U . "■ 
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(!) 

U) (!) 1 1 

(o) (i) (2) By Pascal's idenliiy: 1 ‘ 1 

(0(0(0 0) (iMSM!) * 3 » ' 

0 ) 0 ) 0 ) 0 ) 0 ) . 464 , 

(!) (!) (!) (!) (!) (!) . 5 ,0 10 5 > 

(5) (!) (!) (!) (!) (!) (!) 7 6 15 20 l5 x / 1 
(!) (!) (!) (!) (!) (!) (!) (!) > 7 21 » » 21 7 
(!) (!) (!) (!) (!) (!) (!) (!) (!) ] •»»»*» 1 


1 


m3 (hi 

FIGURE 1 Pascal’s Triangle. 


This tnangle is known as Pascal’s triangle. Pascal’s identity shows that when two 
w gfi adjacent binomial coefficients in this triangle are added, the binomial coefficient in the 
ne xt row 1 between these two coefficients is produced. 

The binomial coefficients enjoy many other identities besides Pascal’s identity. Two 
other identities will be stated here. Combinatorial proofs will be given. Others may be 
found in the exercises at the end of the section. 


THEOREM 4 Let n be a positive integer. Then 

tc(n,k) = 2\ 

k = 0 


Proof: A set with n elements has a total of 2" different subsets. Each subset has either 
zero elements, one element, two elements,.... or n elements in it. There are C(n, 0) 


Blaise Pascal (162.M 662). Blaise Pascal exhibited his talents at an early age, although his father, who 
had made discoveries in analytic geometry, kepi mathematics books away from him to encourage other 
interests. At 16 Pascal discovered an important result concerning conic sections. At IS he designed a cal¬ 
culating machine, which he built and sold, Pascal, along w ith Fermat, laid the foundations for the modem 
theory nf probability, In this work he made new discoveries concerning what is now called Pascal's tnangle, 
In 1654, Pascal abandoned his mathematical pursuits to devote himself to theology, After this, he returned 
to mathematics only once. One night, distracted by a severe toothache, he sought comfort by studying the 
mathematical properties of the cycloid. Miraculously, his pain subsided, which he took as a sign of divine 
approval of the study of mathematic e 
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THEOREM 5 


M'le# 


EXAMPLE 9 


web 


subsets with zero elements, C(n, I) subsets with one element, C(n, 2) subsets with two 
elements,,,,, and C(n, n) subsets with n elements. Therefore, 

VC(rU) 

* — 0 

counts the total number of subsets of a set with n elements. This shows that 

n 

y C(n, k) = 2". 

t=o □ 


vAsimwWhs iDonm l - -g- k^n w*k r 

bid e i-.^rdLnn fid*/ m « TV* 


Qtn - A r\ - V Ct* r - U 

iMAfc Tbs I 

*IT*wdf ip Ijtc dgUfartrifc, mnay, 


Proof: Suppose that there are m items in one set and n items in a second set. Then the 
total number of ways to pick r elements from the union of these sets is Cfm + n, r). 
Another way to pick r elements from the union is to pick k elements from the first set 
and then r - k elements from the second set, where it is an integer with <)</:£ r. 
This can be done in C(m, k)C(n, r - k) ways, using the product rule, Hence, the total 
number of ways to pick r elements from the union also equals 

r 

C(m + n, r) = V C(m, r - k)C(n, k). 

t=o □ 


This proves Vandermonde’s identity. 


THE BINOMIAL THEOREM 


The binomial theorem gives the coefficients of the expansion of powers of binomial 
expressions. A binomial expression is simply the sum of two terms, such as x + y. 
(The terms can be products of constants and variables, but that does not concern us 
here.) The following example illustrates why this theorem holds, 


The expansion of u + v) 3 can be found using combinatorial reasoning instead of mul¬ 
tiplying the three terms out. When (x + >-) 3 = (x + y)(x + y)U + y) is expanded, all 


Alexandre-Theophile Vandermonde (1735-1796). Because Alexandre-Thtophile Vandermonde was a 
sickly child, his physician father directed him to a career in music. However, he later developed an interest 
in mathematics. His complete mathematical work consists of four papers published in 1771-1772, These 
papers include fundamental contributions on the roots of equations, on the theory of determinants, and on 
the knight's tour problem (introduced in the exercises in Section 7,5). Vandermonde's interest in mathe¬ 
matics lasted for only 2 years. Afterward, he published papers on harmony, experiments with cold, and the 
manufacture of steel. He also became interested in politics, joining the cause of the French revolution and 
holding several different positions in government. 
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THEOREM 6 


EXAMPLE 10 


EXAMPLE 11 


products of a term in the first sum, a term in the second sum, and a term in the third 
sum are, added. Terms of the form ,r J . x 2 y, jcy~, and y 3 arise. To obtain a term of the 
form ,r\ an x must be chosen in each of the sums, and this can be done in only one 
way. Thus, the ,c term in the product has a coefficient of 1. To obtain a term of the 
form jry, an x must be chosen in two of the three sums (and consequently a y in the 
other sum). Hence, the number of such terms is the number of 2-combinations of three 
objects, namely. C{3,2). Similarly, the number of terms of the form xy 2 is the number 
of ways to pick one of the three sums to obtain an x (and consequently take a y from 
each of the other two terms). This can be done in C{3,1) w ays. Finally, the only way 
to obtain a y 3 term is to choose the y for each of the three sums in the product, and this 
can be done in exactly one way. Consequently, it follows that 

U * y) 3 = r f 3.r 2 y + 3xy z + y 3 . ■ 


The binomial theorem will now be stated. 


THE BINQMIAL THEOREM Let x and y be variables, and let« be a positive 
frrteg^Titea 

A 

(x+y? =^dn,j)x*- i y J 

;■= o . 

* (!)*■ + +Qr-V + ■ ■ • + ( n f + Qy. 

Proof: A combinatorial proof of the theorem will be given. The terms in the product 
when it is expanded are of the form x"~ J y J for j - 0, 1, 2,.... n. To count the number 
of terms of the form x n ~ J yK note that to obtain such a term it is necessary to choose 
n - j .rs from the n sums (so that the other j terms in the product are vs). Therefore, the 
coefficient of x n ~ J y- 1 is C(n, n - j) = C{n, j). This proves the theorem. □ 

The use of the binomial theorem is illustrated by the following examples. 


What is the expansion of (x + y) 4 ? 

Solution: From the binomial theorem it follows that 

4 

for + y) 4 = yc<4,j)x l -ly’ 

/=« 

- O4.0)a 5 + C(4, \)x } y + C(4, 2)x 2 y 2 + C(4. 3>< v + C(4.4)y 4 

- x A + 4,v 3 y + 6-V 3 y “ + 4,vy 3 -f y 4 . ■ 

What is the coefficient of r l2 y B in the expansion of (x -f y) 15 ? 

Solution: From the binomial theorem it follows that this coefficient is 

C(25, 13) - = 5,200,300. 

13!12! ■ 
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EXAMPLE 12 What is the coefficient of je ! V 3 in the expansion of (2x - 3y) 2S ? 

Solution; First, note that this expression, equals (2jc + (—3y)) 2S . By the binomial the¬ 
orem, we have 

25 

(It + (-3 y)) 25 - yC(25J)(2xf s -U-3yy. 

Consequently, the coefficient of in the expansion is obtained when j - 13. 

namely, 

C<25, 13)2°(-3) 13 - - y ^ T 2 12 3 13 . B 


The binomial theorem can be used to give another proof of Theorem 4. Recall that 
this theorem states that V*. „ C(n, k) - 2 n whenever n is a positive integer. 

Proof; Using the binomial theorem we see that 

2" = (1 + if = COU )!*! n ~ k = f C(n, k), 

k -0 *=0 


This is the desired result, □ 

The binomial theorem can also be used to prove the following identity. 


THEOREM 7 Let n be a positive integer. Then 

2>I)*C(«,L) = 0. 


Proof: From the binomial theorem it follows that 

n n 

0 - f(-l)+ 1)" = ^C(n, k)(-l) k \ n ~ k - Tc'O, kf-Xf. 
t-^o r--o 

This proves the theorem. 


□ 


Exercises 

1. List all the permutations of {a, b. c}. 

2. How many permutations are there of" the set 

{a. b, c, d, e, f, g}1 

3. How many permutations of [a, b, c, d. e. f, end with 
o'? 

4. Let S •= {1,23.4,51, 

a) List all the 3 -permutations of S. 
t>l List all the 3 combinations of 5. 


5. Find the value of each of the following quantities, 

a) TO. 3) b) P(6, 5) c) f(8,l) 

d) F(«,5) t) P( 8,8) 0 P(!0.9J 

6. Find the value of each of the following quantities, 

a) C(5,1) b) 05, 3) c) C(8,4) 

d) C{8, 8) e) OS. 0) f) Cl 12,6) 

7. Find the number of S-pemunations of a set with nine 
elements, 


PDF created with pdfFactory trial version www.pdffactorv.com 





258 4 / Counting 


8. In how many different orders can five runners finish a 
race if no ties are allowed'? 

9. How many possibilities are there for the win, place, and 
show (first, second, and third) positions in a horse race 
with 12 horses if all orders of finish are possible? 

1(1. There are six different candidates for governor of a 
state. In how many different orders can the names of 
the candidates be printed on a ballot? 

11. A group contains n men and n women. How many ways 
are there to arrange these people in a row if the men and 
women al tern ale? 

12. In how many ways can a set of two positive integers 
less than !00 be chosen? 

13. In how many ways can a set of five letters be selected 
from the English alphabet? 

14. How many subsets with an odd number of elements 
does a set with 10 elements have? 

15. How many subsets with more than two elements does 
a set with 100 dements have? 

16. How many bit strings of length 30 have 

a) exactly three 0s' ! 

b) the same number of Os as Is? 

c) at least seven Is? 

d) at least three 1 s' 1 

17. One hundred tickets, numbered 1,2,3.100, are 

sold to 100 different people for a drawing. Four dif¬ 
ferent prizes are awarded, including a grand prize (a 
trip to Tahiti). How many ways are there to award the 
prizes if 

a) there are no restrictions? 

b) the person holding ticket 47 wins the grand prize? 

c) the person holding ticket 47 wins one of the prizes? 

d) the person holding ticket 47 does not win a prize? 
el the people holding tickets 19 and 47 both win 

prizes' 1 

f) the people holding tickets 19, 47, and 73 all win 
prizes? 

g) the people holding tickets 19,47,73, and 97 all win 
prizes? 

h) none of the people holding tickets 19, 47. 73, and 
97 wins a prize? 

i) the grand prize winner is a person holding ticket 19, 

47, 73, or 97? 

j) the people holding tickets 19 and 47 win prizes, 
but the people holding tickets 73 and 97 do not win 
prizes? 

IS. Thirteen people on a so ft bail team show up for a game, 

a) How many ways are there to choose 10 players to 
take the field? 

b 3 How many ways are there to assign the I 0 positions by 
selecting players from the 13 people who show up? 

e) Of the 13 people who show up, 3 are women. Hew 
many ways ate there to choose 10 players to take 
liie Held if at kwi one of these players must be a 
woman ’ 


19. A club has 25 members. 

a) Hem many ways are there to choose four members 
of the club to .serve on an executive committee? 

b) How many ways are there to choose a presi¬ 
dent, vice president, secretary, and treasurer of the 
club? 

20. A professor writes 40 discrete mathematics true/false 
questions. Of the statements in these questions, 17 are 
true. If the questions can be positioned in any order, 
bow many different answer keys arc possible? 

21. How many 4-permutations of the positive integers not 
exceeding 100 contain three consecutive integers in the 
correct order 

a) where consecutive means in the usual order of the 
integers and where these consecutive integers can 
perhaps be separated by other integers in the per¬ 
mutation? 

b) where consecutive means both that the numbers be 
consecutive integers and that they be in consecutive 
positions in the permutation? 

22. Seven women and nine men are on the faculty in the 
mathematics department at a school, 

a) How many ways are there to select a committee 
of five members of the department if at least one 
woman must be on the committee? 

b) How many ways are there hi select a committee 
of five members of the department if at least one 
woman and at least one man must be on the com¬ 
mittee? 

23. The English alphabet contains 21 consonants and five 
vowels. How many strings of six lowercase letters of 
the English alphabet contain 

a) exactly 1 vowel? 

b) exactly 2 vowels? 

e) at least 1 vowel? 
d) at least 2 vowels? 

24. How many strings of six lowercase letters from the En¬ 
glish alphabet contain 

a) the letter«? 

b) the letters a and >•'' 

c) the letters a and b in consecutive positions with a 
preceding b. with all the letters distinct? 

d> the letters o and b. where a is somew here to the left 
of b in the string, with all the letters distinct? 

25. Suppose that a department contains 10 men anti 15 
women. How many ways are there to form a commit¬ 
tee with 6 members if it must have the same number of 
men and women? 

26. Suppose that a department contains 30 men and 15 
women. How many ways are there to form a commit¬ 
tee with ft members if it must have more women than 
men? 

27. How many bit strings contain exactly eight 0s and ten 
Is if every 0 must be immediately followed by a T> 
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28, How many bit strings contain exactly live Os and four¬ 
teen Is if every 0 must be immediately followed by 
two Is? 

29, How many bit strings of length 10 contain at least three 
1 s and at least three Os'? 

30, How many ways are there to select 12 countries in the 
United Nations to serve on a council if 3 are selected 
from a block of 45, 4 arc selected from a block of 57, 
and the others are selected from the remaining 69 Coun¬ 
tries'.’ 

31, How many license plates consisting of three letters fol¬ 
lowed by three digits contain no letter or digit twice? 

32, How many ways are there to seat six people around a 
circular table, where searings are considered to be the 
same if they can be obtained from each other by rotat¬ 
ing the table? 

33, Show that if n and k are positive integers, then 

C(« t- U) = (0 4 l)C(n, k - 1 )/fc 

Use this identity to construct an inductive definition of 
the binomial coefficients, 

34, Show that if p is a prime and k is an integer such that 
1 if k s p — 1, then p divides C{p, k). 

35, Find the expansion of ft + y) s . 

36, Find the coefficient of r 5 y* m (,v 4 y) i3 . 

37, How manv terms are there in the expansion of (.r 4 
vjftny 

38, What is the coefficient of x 1 in (1 -t- x) 1! ? 

39, What is the coefficient of x 9 in (2 - ,r) 19 ? 

40, What is the coefficient of ** v 9 in the expansion of (3x4 

2y) 17 ? 

41, What is the coefficient of x 10I y' w in the expansion of 
(2 a ■■ 3y}«? 

*42. Giv e a formula for the coefficient of in the expansion 
of ( t — l/.r) , where k is an integer. 

*43, Give a formula for the coefficient of jt* in the expansion 
of i .v 7 - i/.t) 100 . where k is an integer, 

44. The row of Pascal’s triangle containing the binomial 
coefficients Cf 10, 1), 0 a k -4 10, is: 

1 It) 45 120 210 252 210 120 45 10 I 

Use Pascal s identity to produce the row immediately 
following this tow in Pascal’s triangle. 

45. What is the row of Pascal’s triangle containing the bi¬ 
nomial coefficients C(9, i),0 s k s 9? 

*46, La a be a positive integer. What is the largest binomial 
coefficient C{n, r), where r is a nonnegati vc integer less 
than or equal to n? Prove your answer is correct. 

47. Prove Pascal’s identity, using the formula for C(n, r). 

48. Prove the identity C(n, r)C(r, k) - C(n, k) ■ C(n - 
k,r- k), whenever n, t , and k are nonnegative integers 
with r < n and k _■£ r, 

a) using a combinatorial argument. 

b) using an argument based on the formula for the 
number of /--combinations of a set with n elements. 


*49, Prove that 

’ "y' C(n 4 k, k} - C(n 4 r 4 1, r), 

t=o 

whenever n and r are positive integers, 

a) using a combinatorial argument. 

b) using Pascal’s identity, 

50, Show that if n is a positive integer, then C(2n,2) - 
2 C(n, 2} + n 1 

a) using a combinatorial argument. 

b) by algebraic manipulation, 

*51. Give a combinatorial proof that X"= kCut k) = 
n'2 !> 1 , (Hint: Count in two ways the number of ways 
to select a committee and to then select a leader of the 
committee.) 

*52. Give a combinatorial proof thaL y.J,] kC{n,kf - 
nCiln -1, it— I). [Hint: Count in two ways the number 
of ways to select a committee, with n members from a 
group of n mathematics professors and n computer sci¬ 
ence professors, such that the chairperson of the com¬ 
mittee is a mathematics professor.) 

53. Show that a set has the same number of subsets with 
an odd number of dements as it does subsets with an 
even number of elements. 

*54. Prove the binomial theorem using mathematical induc¬ 
tion. 

55. In this exercise we will count the number of paths in the 
xy plane between the origin (0,0) and point (m.n) such 
that each path is made up of a scries of steps, where 
each step is a move one unit to the right or a move one 
unit upward. (No moves to the left or downward are 
allowed.) Two such paths from (0,01 to (5, 3) are illus¬ 
trated here. 



















(b) 


(5,3) 


a) Show that each path of the type described can be 
represented by a bit string consisting of m Os and n 
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1 s, where a 0 represents a move one unit to the right 
and a l represents a move one unit upward, 

b) Conclude from pari (a) that there are C(m + run) 
paths of the desired type. 

56. Use Exercise 55 to prove that C(n, k) = C(n, n - k) 
whenever k is an integer with 0 s H x [Him: Con¬ 
sider the number of paths of the type described in Ex¬ 
ercise 55 from (0, 0) to (n - k, k) and from (0.0) to 
i k, n - £).] 

57. Use Exercise 55 to prove Theorem 4, [Hint: Count die 
number of paths with n steps of the type described in 
Exercise 55. Every such path must end at one of the 
points (n - k, k} for k ■= 0. 1, 2,, n.j 

58. Use Exercise 55 to prove Pascal’s identity. [Hint: Show' 
that a path of the type described in Exercise 55 from 
(0, 0) to (n + 1 - k, k) passes through either (n + 1 - 
k, k - 1) or v’i - k, k). but not through both.] 

59. Prove the identity in Exercise 49 using Exercise 55, 

I Him: First, note that the number of paths from (0,0) to 
In + 1. r) equals C(rt +1 + r, r). Second, count the num- 
berof paths by summing the number of these paths that 
start bv going k units upward for k = 0,1,2.r ] 

*61). How many ways are there for a horse race with four 
horses to finish if ties are possible? (Note that since ties 
are allowed, any number of the four horses may tie.) 

*61. There are six runners in the 100-yard dash. How many 
ways are there for three medals to be awarded if 
tics are possible? (The runner or runners who finish 
with the fastest time receive gold medals, the runner 
or runners who finish with exactly one runner ahead 
receive silver medals, and the runner or runners who 
finish with exactly two runners ahead receive bronze 
medals.) 

*62. The following procedure is used to break ties in games 
in the championship round of the World Cup soccer 


tournament. Each team selects five players in a pre¬ 
scribed order. Each of these players takes a penalty 
kick, with a player from the first team followed by a 
player from the second team and so on, following the 
order of players specified. If the score is still tied at the 
end of the ten penalty kicks, this procedure is repeated, 
If the score is till tied after 20 penalty' kicks, a sudden- 
death shootout occurs, with the first team scoring an 
unanswered goal victorious. 

a) ETow many different scoring scenarios are possible 
if the game is settled in the lint round of 10 penalty 
kicks, where the round ends once it is impossible 
for a team to equal the number of goals scored by 
the other team? 

b) How many different scoring scenarios for the first 
and second groups of penalty kicks are possible if 
the game is settled in the second round of 10 penalty 
kicks? 

c) How many scoring scenarios are possible for the 
full set of penalty kicks if the game is settled with 
no more than 10 total additional kicks after the two 
rounds of 5 kicks for each team? 

*63. Determine a formula involving binomial coefficients 
for the nth term of a sequence if its initial terms are 
those listed. (Hint: Looking at Pascal’s triangle will be 
helpful. Although infinitely many sequences start with 
a specified set of terms, each of the following lists is 
the start of a sequence of the type desired.) 

a) 1,3,6, 10,15, 21, 28, 36,45, 55, 66,... 

b) 1,4,10, 20, 35,56, 84, 120, 165, 220,... 

c) 1,2, 6.20.70,252,924,3432, 12870,48620,... 

d) 1,1, 2,3, 6,10,20,35,70, 126,... 

e) 1, I. 1,3, 1,5, 15,35, 1.9,... 

f) 1, 3, 15, 84, 495, 3003, 18564, 116280, 735471, 
4686825,.,, 


44 __ 

Discrete Probability 

INTRODUCTION 


Combinatorics and probability theory share common origins. The theory of probability 
was first developed in the seventeenth century when certain gambling games were an¬ 
alyzed by the French mathematician Blaise Pascal. It was in these studies that Pascal 
discovered various properties of the binomial coefficients. In the eighteenth century, 
the French mathematician Laplace, who also studied gambling, gave a definition of the 
probability of an event as the number of successful outcomes divided by the number of 
possible outcomes. For instance, the probability a die comes up an odd number when 
it is rolled is ihe number of successful outcomes—namely, the number of ways it can 
crime up odd- divided by the number of possible outcomes—namely, the number of 
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EXAMPLE 1 


EXAMPLE 2 


different ways the die can come up. There are a total of sis possible outcomes—namely, 
1,2, 3. 4,5, and 6—and exactly three of these are successful outcomes—namely, 1,3, 
and 5. Hence, the probability that the die comes up an odd number is 3/6 - 1/2. (Note 
that it has been assumed that all possible outcomes are equally likely, or, m other words, 
that the die is fair.) 

In this section we wall restrict ourselves to experiments that have finitely many, 
equally likely, outcomes. This permits us to use Laplace’s definition of the probability 
of an event. We will continue our study of probability in Section 4.5, where we will 
study experiments with finitely many outcomes that are not. necessarily equally likely. 
In Section 4.5 we will also introduce some key concepts in probability theory, including 
conditional probability, independence of events, random variables, and expected values. 


FINITE PROBABILITY 


An experiment is a procedure that yields one of a given set of possible outcomes. The 
sample space of the experiment is the set of possible outcomes. An event is a subset of 
the sample space. Laplace’s definition of the probability of an event with finitely many 
possible outcomes will now be stated. 


DEFINITION 1. Ttut probability of as event E, which is a subset of a finite sample 
space S of equally likely outcomes, is m - wm- 


Some additional examples are given here. 


An urn contains four blue balls and five red balls. What is the probability that a ball 
chosen from the urn is blue? 

Solution: To calculate the probability, note that there are nine possible outcomes, and 
four of these possible outcomes produce a blue ball. Hence, the probability that a blue 
ball is chosen is 4/9. ■ 


What is the probability that when two dice are rolled, the sum of the numbers on the 
two dice is 7? 


mb 


Pierre Simon Laplace (1749-1827), Pierre Simon Laplace came from humble origins in Normandy. In 
his childhood he was educated in a school run by the Benedictines. At 16 he entered the University of 
Caen intending to study theology. However, he soon realized his true interests were in mathematics. After 
completing his studies he was named a provisional professor at Caen, and in 1769 he became professor of 
mathematics at the Paris Military School. 

Laplace is best known for his contributions to celestial mechanics, the study of the motions of heav¬ 
enly bodies. His Traite du Micaniqm Celeste is considered one of the greatest scientific works of the early 
nineteenth century. Laplace was one of the founders of probability theory and made many contributions 
to mathematical statistics. His work in this area is documented in his book Thkorie Amlytique des Proba¬ 
bilities, fr which he defined the probability of an event a.s the ratio of the number of fav orable outcomes to 
the total number of outcomes of an experiment. 

Laplace was famous for his political flexibility. He was loyal, in succession, to the French Republic. 
Napoleon, and King Louis XVIII. This permitted him to be productive before, during, and after the French 

revolution. 
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W'4 

Solution: There are a total of 36 possible outcomes when two dice are rolled. (The 
product rule can be used to see this; since each die has 6 possible outcomes, the to¬ 
tal number of outcomes when two dice are rolled is 6 2 = 36.) There are 6 successful 
outcomes, namely, (1, 6), (2,5), (3, 4), (4, 3), (5, 2), and (6, 1), where the values of the 
first and second dice are represented by an ordered pair. Hence, the probability that a 7 
comes up when two fair dice are rolled is 6/36 =1/6, ■ 

Lotteries have become extremely popular recently. We can easily compute the odds 
'■ of winning different types of lotteries. 

EXAMPLE 3 

In a lottery, players win a large prize when they pick four digits that match, in the correct 
order, four digits selected by a random mechanical process. A smaller prize is won if 
only three digits are matched. What is the probability that a player wins the large prize? 
What is the probability that a player wins the small prize? 

Solution: There is only one way to choose all four digits correctly. By the product rule, 
there are 10 4 = 10,000 ways to choose four digits. Hence, the probability that a player 
wins the large prize is 1/10,000 - 0.0001. 

Players win the smaller prize when they correctly choose exactly three of the four 
digits. Exactly one digit must be wrong to get three digits correct, but not all four correct. 
By the sum rule, the number of ways to choose exactly three digits correctly can be 
obtained by adding the number of ways to choose four digits matching the digits picked 
in all but the fth position, for i = 1,2.3, 4. To count the number of successes with the 
first digit incorrect, note that there are nine possible choices for the first digit (all but 
the one correct digit), and one choice for each of the other digits, namely, the correct 
digits tor these slots. Hence, there are nine ways to choose four digits where the first 
digit Is incorrect, but the last three are correct. Similarly, there are nine ways to choose 
four digits where the second digit is incorrect, nine with the third digit incorrect, and 
nine with the fourth digit incorrect. Hence, there is a total of 36 ways to choose four 
digits with exactly three of the four digits correct. Thus, the probability that a player 
wins the smaller prize is 36/10,000 = 9/2500 = 0.0036. ■ 

EXAMPLE 4 

mb 

There are many lotteries now that award enormous prizes to people who correctly 
choose a set of six numbers out of the first n positive integers, where n is usually be¬ 
tween 30 and 50. What is the probability that a person picks the correct six numbers 
out of 40? 

Solution: There is only one winning combination. The total number of ways to choose 
six numbers out of 40 is 

Ct 40,6) - = 3,838,380. 

34!6! 

Consequently, the probability of picking a winning combination is i/3,838.380 ~ 
0.00000026. (Here the symbol — means approximately equal to.) ■ 

1 We can find the probability of hands in card games using the techniques developed 

so tar. A deck of cards contains 52 cards. There are 13 different kinds of cards, with 4 
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EXAMPLES 


EXAMPLE 6 


EXAMPLE 7 


THEOREM 1 


cards of each kind. These kinds are twos, threes, fours, fives, sixes, sevens, eights, nines, 
tens, jacks, queens, kings, and aces. There are also four suits, spades, dubs, hearts, and 
diamonds, each containing 13 cards, with one card of each kind in a suit. 


How many different hands of 5 cards from the deck of 52 are there? 

Solution: There are C(52, 5) - 2,598,960 different hands with five cards. ■ 


Find the probability that a hand of live cards in poker contains four cards of one kind. 


Solution: By the product rule, the number of hands of five cards with four cards of one 
kind is the product of the number of ways to pick one kind, the number of ways to pick 
the four of this kind out of the four in the deck of this kind, and the number of ways to 
pick the fifth card. This is 

C(13, l)C(4,4)C(4M). 

Since there is a total of C(52,5) different hands of five cards, the probability that a hand 
contains four cards of one kind is 


C(13, 1}C(4,4)C(48,1} 
£7(52, 5) 


13- 148 
2,598.960 


0.00024. 


What is the probability that a poker hand contains a full house, that is, three of one kind 
and two of another kind? 


Solution: By the product rule, the number of hands containing a full house is the prod¬ 
uct of the number of ways to pick two kinds in order, the number of ways to pick three 
out of four for the first kind, and the number of ways to pick two out of four for the sec¬ 
ond kind, (Note that the order of the two kinds matters, since, for instance, three queens 
and two aces is different than three aces and two queens.) We see that the number of 
hands containing a full house is 

P(I3,2)C(4,3)C(4,2) = 13 ■ 12 • 4 ■ 6 = 3744. 


Since there are 2,598,960 poker hands, the probability of a full bouse is 


3744 

25984)60 


0.0014. 


THE PROBABILITY OF COMBINATIONS OF EVENTS 

We- can use counting techniques to find the probability of events derived from other 
events. 

Let £ be an event in a sample space S. The probability of the event E, the comple¬ 
mentary event of E, is given by 

me) -1 ••• pm. 
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EXAMPLE 8 


THEOREM 2 


Proof: To find the probability of the event £, note that }£| = |S| - |£|. Hence, 

p(f) * nTr = 1 ■ | = 1 ■ m □ 

There is an alternative strategy for finding the probability of an event when a direct 
approach does not work well. Instead of determining the probability of the event, the 
probability of its complement can be found. This is often easier to do. as the following 
example shows. 


A sequence of 10 bits is randomly generated. What is the probability that at least one 
of these bits is 0? 

Solution: Let E he the event that at least one of the 10 bits is 0. Then E is the event 
that all the bits are i s. Since the sample space S is the set of all bit strings of length 10, 


it follows that 


P(E) = 1 

- P(E) 

— 1 

i e: 

— 1 

!s[ 

— 1 

1 

— 1 

~ pi 

= 1 

1 

~~ 1024 


1023 
' 7024' 

Hence, the probability that the bit string will contain at least one 0 bit is 1023/1024. It 
is quite difficult to find this probability directly without using Theorem 1. ■ 


We can also find the probability of the union of two events. 


Let E[ and £2 be events in the sample space S. Then 
P<E 1 U E 2 ) p(E { ) + p(E 2 ) ~ p{Ei n £ 2 ). 


Proof: Using the formula given in Section 1,4 for the number of elements in the union 
of two sets, it follows that 

|£, u £ 2 [- : £i | + |Li|- |£,n&i. 


Hence, 

/j(£i U £ 2 ) 


j£ j u £ 2 | 

(SI 

| £i) + l£ 2 | - |£i H£ 2 j 

\S\ 


[Ei] fel lEiEfEf 

fSl \S\ ur 


= p(£i) + piEi) - p(E] n £;). 


□ 
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EXAMPLE 9 

What is the probability' that a positive integer selected at random from the set of positive 
integers not exceeding 100 is divisible by either 2 or 5? 

Solution: Let E\ be the event that the integer selected is divisible by 2, and let Ej be 

the event that it is divisible by 5. Then E] U is the event that it is divisible by either 

2 or 5. Also, £, n £2 is the event that it is divisible by bath 2 and 5, or equivalently, that 
it is divisible by 10. Since £j - 50, |£ 2 = 20, and £ H £j| - 10, it follows that 

p(E] U Ei) ~ p(E\ ) + p(E 2 ) - p(E l n £2) 

50 20 10 

~ Too + 100 Too 
_ 3 

“ 5 "' ■ 

PROBABILISTIC REASONING 

A common problem is to determine which of two events is more likely. Analyzing the 
probabilities of such events can be tricky. The following example describes a problem 
of this type. It discusses a famous problem originating with the television game show 
“Let’s Make a Deal.” 

EXAMPLE 10 

The Monty Hall Three Door Puzzle. Suppose you are a game show contestant. You 


have a chance to win a large prize. You are asked to select one of three doors to open; the 
larger prize is behind one of the three doors. Once you select a door, the game show host, 
who knows what is behind each door, does the following. First, whether or not you se¬ 
lected the winning door, he opens one of the other tw o doors that he knows is a losing door 
(selecting at random if both are losing doors). Then he asks you whether you would like 
to switch doors. Which strategy should you use? Should you change doors or keep your 
original selection, or does it not matter? 

Solution: The probability you select the correct door (before the host opens a door and 
asks you whether you want to change) is J/3, since the three doors are equally likely 
to be the correct door. The probability this is the correct door does not change once the 
game show host opens one of the other doors, since he will always open a door that the 
prize is not behind. 

The probability that you selected incorrectly is the probability the prize is behind 
one of the two doors you did not select. Consequently, the probability that you selected 
incorrectly is 2/3. If you selected incorrectly, when the game show host opens a door 
to show you that the prize is not behind it, the prize is behind the other door, You will 
always win if your initial choice was incorrect and you change doors. So, by changing 
doors, the probability you win is 2/3. In other words, you should always change doors 
when given the chance to do so by the game show host. This doubles the probability 
that you will win. ■ 

Exercises 

-—-—----—--—- 


1* What is the probability that a card selected from a deck 2. What is the probability that a die comes up six when 
is an ace? is rolled? 
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3. What is the probability that a randomly selected integer 
chosen from the first 100 positive integers is odd? 

4. What is the probability that a randomly selected day of 
the year (from the 366 possible days) is in April? 

5. What is the probability that the sum of the numbers on 
two dice is even when they are rolled? 

6. What is the probability that a card selected from a deck 
is an ace or a heart? 

1. What is the probability that a coin lands heads up six 
times in a row? 

8. What is the probability that a five-card poker hand con¬ 
tains the ace of hearts'.' 

9. What is the probability that a five-card poker hand does 
not contain the queen of hearts ? 

10. What is the probability that a five-card poker hand con¬ 
tains the two of diamonds and the three of spades? 

11. What is the probability that a five-card poker hand con¬ 
tains the two ol diamonds, the three of spades, the sis 
of hearts, the ten of dubs, and the king of hearts? 

12. What is the probability that a five-card poker hand con¬ 
tains exactly one ace? 

13. What is the probability that a five-card poker hand con¬ 
tains at least one ace? 

14. What is the probability that a five-card poker hand con¬ 
tains cards of live different kinds? 

15. What is the probability that a five-card poker hand con¬ 
tains two pairs (that is, two of each of two different 
kinds and a fifth card of a third kind)? 

16. What is the probability that a five-card poker hand con¬ 
tains a flush, that is, five cards of the same suit? 

17. What is the probability that a five-card poker hand con¬ 
tains a straight, that is, five cards that have consecutive 
kinds? (Note that an ace can be considered either the 
lowest card of an A-2-3-4-5 straight or the highest card 
of a (0-J-Q-K-A straight.) 

18. What is the probability that a five-card poker hand con¬ 
tains a straight flush, that is, five curds of the same suit 
of consecutive kinds? 

*19. What is the probability that a five-card piker hand con¬ 
tains cards of five different kinds and does not contain 
a flush ora straight? 

20. What is the probability that a five-card poker hand con¬ 
tains a royal flush, that is, the 10, jack, queen, king, and 
ace of one suit? 

21. What is the probability that a die never conies up an 
even number when it is rolled six times' 1 

22. What is the probability that a positive integer not ex¬ 
ceeding 100 selected at random is divisible by 3? 

23. What is the probability that a positive integer not ex¬ 
ceeding 100 selected at random is divisible by 5 or 7? 

24. Find the probability of winning the lottery by selecting 
the correct six integers, where the order in which these 
integers arc selected does not matter, from the positive 
integers not exceeding 

a) 30. hi 36. c) 42. d) 48. 


25. Find the probability of winning the lottery by selecting 
the correct six integers, where the order in which these 
integers are selected does not matter, from the positive 
integers not exceeding 

a) 50, b) 52. c) 56, d) 60. 

26. Find the probability of selecting none of the correct six 
integers, where the order in which these integers are 
selected does not matter, from the positive integers not 
exceeding 

a) 40. b) 48. c) 56. d) 64. 

27. Find the probability of selecting exactly one of the cor¬ 
rect six integers, where the order in which these in¬ 
tegers arc selected decs not matter, from the positive 
integers not exceeding 

a) 40, b) 48. c) 56, d) 64. 

28. To play the Pennsylvania superlottery, a player selects 
7 numbers out of the first 80 positive integers. What is 
the probability that a person wins the grand prize by 
picking 7 numbers that are among the 11 numbers se¬ 
lected by the Pennsylvania lottery commission? 

29. In a superlottery, players win a fortune if they choose 
the eight numbers selected by a computer from the pos¬ 
itive integers not exceeding 100. What is the probabil¬ 
ity that a player wins this superlottery? 

30. What is the probability that a player wins the prize of¬ 
fered for correctly choosing five (but not six) numbers 
out of six integers chosen between 1 and 40, inclusive, 
by a computer? 

31. In roulette, a wheel with 38 numbers is spun. Of these, 
18 are red, and 18 are black. The other two numbers, 
which are neither black nor red, are 0 and 00. The prob¬ 
ability that when the wheel is spun it lands on any par¬ 
ticular number is 1/38. 

a) What is the probability that the wheel lands on a red 
number? 

b) What is the probability that the wheel lands on a 
black number twice in a row? 

c) What is the probability that the wheel lands on 0 or 
00 ? 

d) What is the probability that the wheel dives not land 
on 0 or 00 five times in a row? 

e) W'hat is the probability that the wheel lands on a 
number between 1 and 6, inclusive, on one spin, 
but does not land between them on the next spin? 

32. Which is more likely, rolling a total of 8 when two dice 
are rolled or rolling a total of 8 when three dice are 
rolled? 

33. Which is more likely, rolling a total of 9 when two dice 
are rolled or rolling a total of 9 when three dice are 
rolled? 

34. Two events £? and £? are called independent if p{E\ n 
£j) - P(£i)p(Ei)< For each of the following pairs of 
events, which are subsets of the set of all possible out¬ 
comes when a coin is tossed three times, determine 
whether or not they are independent. 
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a) Ej: the first coin comes up tails; E 2 : the second coin 
comes up heads. 

b) E\: the first coin comes up tails; Ev. two, and not 
three, heads come up in a row, 

c) E[ : the second coin comes up tails; E 2 : two, and not. 
three, heads come up in a row. 

(We will study independence of events in more depth 
in Section 4.5.) 

35. Explain what is wrong with the statement that in the 
Monty Hall Three Door Puzzle the probability that 
the prize is behind Lire first door you select and the 


probability that the prize is behind the other of the two 
doors that Monty does not open are both 1/2, since there 
are two doors left. 

3b. Suppose that instead of three doors, there are four doors 
in the Monty Hail puzzle. What is the probability that 
you win by not changing once the host, who knows 
what is behind each door, opens a losing door and gives 
you the chance to change doors? What is the probabil¬ 
ity that you win by changing the door you select to one 
of the turn remaining doors among the three that you 
did not select? 


4 1 5 _ 

Probability Theory 


INTRODUCTION 


In Section 4.4 we introduced the notion of the probability of an event. (Recall that an 
event is a subset of the possible outcomes of an experiment.) We defined the probability 
of an event E as Laplace did, that is, 


the number of outcomes in E divided by the total number of outcomes. This definition 
assumes that all outcomes are equally likely. However, many experiments have out¬ 
comes that are not equally likely. For instance, a coin may be biased so that it comes up 
heads twice as often as tails. Similarly, the likelihood that the input of a linear search 
is a particular element in a list, or is not in the list, depends on how the input is gener¬ 
ated. How can we model the likelihood of events in such situations? In this section we 
will show how to define probabilities of outcomes to study probabilities of experiments 
where outcomes may not be equally likely. 

Suppose that a fair coin is flipped four times, and the first time it comes up heads 
Given this information, what is the probability that heads comes up three times? To 
answer this and similar questions, wc will introduce the concept of conditional probo- 
bitm. Does knowing that the first flip comes up heads change the probability that heads 
comes up three times? It not, these two events are called independent, a concept studied 
later in this section. 

Many questions address a particular numerical value associated with the outcome 
of an experiment. For instance, when we flip a coin 100 times, what is the probability 
that exactly 40 heads appear? How many heads should we expect to appear? In this 
section we will study random variables, which axe functions that associate numerical 
values to the outcomes of experiments, and their weighted averages, called expected 
value. r. 

ASSIGNING PROBABILITIES 


Let S be the sample space of an experiment with a finite or countable number of out¬ 
comes. We assign a probability pfx) to each outcome s. We require dial two conditions 
be met: 
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EXAMPLE 1 


(<) 0 p\s) « L for each seS 
and 

(it) , p(s) - I. 

seS 

Condition (i) states that the probability of each outcome is a nonnegative real number 
no greater than 1. Condition (ii) states that the sum of the probabilities of all possible 
outcomes should be 1; that is, when we do the experiment, il is a certainty that one of 
these outcomes occurs. This is a generalization of Laplace’s definition in which each 
of n outcomes is assigned a probability of \!n. Indeed, conditions (i) and (ii) are met 
when Laplace’s definition of probabilities of equally likely outcomes is used. (See 
Exercise 4.) 

Note when there are n possible outcomes, ,t], .v?.,v n , the two conditions to be 

met are 

0) 0 -- p(Xj ) s 1 for i — 1, 2,,.., « 

and 

n 

(«) p(X,) - l. 

f'-t 

To model an experiment, the, probability p(s) assigned to an outcome i should equal 
the limit of the number of times s occurs divided by the number of times the experiment 
is performed, as this number grows without bound. (We will assume that all experiments 
discussed have outcomes that are predictable on the average, so that this limit exists. 
We also assume that the outcomes of successive trials of an experiment do not depend 
on past results.) 

Remark; In this section we will require that the number of possible outcomes is finite, ,4 
countably infinite number of outcomes can be handled similarly using infinite series, as 
illustrated in Exercises 37-40 at the end of this section. We will not discuss probabilities 
of events when the set of outcomes is not discrete, such as when the outcome of an 
experimem can be any real number. In such cases integral calculus is usually required 
for the study of the probabilities of events. 

We can model experiments in which outcomes are either equally likely or not 
equally likely by choosing the appropriate function pis), as Example 1 illustrates, 


What probabilities should we assign to the outcomes H (heads) and T (tails) when a fair 
coin is flipped? What probabilities should be assigned to these events when the coin is 
biased so that heads comes up tw ice as often as tails? 


Solution: For a fair coin, the probability that heads comes up when the coin is flipped 
equals the probability that tails comes up, so that the events are equally likely. Con¬ 
sequently, we assign the probability 1/2 to each of the two possible outcomes, that is, 
pm) - piX) = 1 / 2 , 

For the biased coin we have 

p(H) ~ 2 pin. 
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EXAMPLE 2 


Since 

P {H) + P m - i, 

it follows that 

2 pCD + pin - 3/7(7 ) - 1. 

We conclude that p(T) = 1/3 and p(H) -- 2/3. ■ 

We now define the probability of an event as tlie sum of the probabilities of the 
outcomes in this event. 


DtiWIION 1. The probability of the event £ is the sian of the probabilities of 
the outcomes in £ That k, 

p(E) - Ypis). 
seE 


Note that when there are n outcomes in the event E, that is, if E - {«], cti ,..., a„\. 
then p(E) - y " =] pi a, ) 


Suppose that a die is biased {or loaded) so that 3 appears twice as often as each other 
number but that the other five outcomes are equally likely. What is tire probability that 
an odd number appears when we roll this die? 

Solution: We want to find the probability of the event E -{1,3,5}. By Exercise 2 at 
the end of this section, we have 

pi 1) - p( 2) = p{ 4) = pin = Pi 6) = 1/7; pO) - 2/7. 

It follows that 

p(E) - p(t) + p( 3) + p(5) - 1/7 + 2/7 + 1/7 - 4/7. ■ 


Whet; events are equally likely and there are a finite number of possible outcomes, 
the definition of the probability of an event given in this section (Definition 1) agrees 
w'ith Laplace’s definition (Definition 1 of Section 4.4), To see this, suppose that there 
are n equally likely outcomes; each possible outcome has probability l In, since the 
sum of their probabilities is 1. Suppose the event E contains m outcomes. According to 
Definition 1, 


p\E) - 


= \ ’ 


m 

n 


Since j£| - m and |£[ - /t, it follows that 


P(E) 


m 

n 


m 
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This is Laplace’s definition of the probability of the event t. 

COMBINATIONS OF EVENTS 

The formulae for probabilities of combinations of events in Section 4.4 continue to 
hold when we use Definition 1 to define the probability of an event. For example. 
Theorem 1 of Section 4.4 asserts that 

p\E) - 1 - p(E) 

where E is the complementary event of the event E. This equality also holds when 
Definition 1 is used. To see this, note that since the sum of the probabilities of the n 
possible outcomes is 1, and each outcome is either in E or in E, but not in both, we have 

V p{s) ■ I - p(E) + piEi. 

i*S 

Hence, p(E) - I - piE). 

Under Laplace's definition, by Theorem 2 in Section 4.4, we have 

p(E\ U E 2 ) - p(£y) + p{Ez) - p(E] n EE) 

whenever E\ and E 2 are events in a sample space S. This also holds when we define 
the probability of an event as we do in this section. To see this, note that p(E\ U Ej) is 
the sum of the probabilities of the outcomes in E\ U £->. When an outcome x is m one, 
but not both, of E\ and IE, p{.x) occurs in exactly one of the sums for piE j) and piEi). 
When an outcome x is in both E\ and Ez, p(x) occurs in the sum for p(E{), in the sum 
for fit-:), and in the sum for p(E [ n EE}, so that it occurs 1 + 1 - 1 = 1 time on file 
right-hand side. Consequently, the left-hand side and right-hand side are equal. 

CONDITIONAL PROBABILITY 


Suppose that we flip a coin three times, and all eight possibilities are equally likely. 
Moreover, suppose we know that the event F. that the first flip comes up tails, occurs 
Given this information, what is the probability of the event E, that an odd number of tails 
appears? Since the first flip comes up tails, there are only four possible outcomes; 717. 
TTH. 77/7, and 77/77. where H and T represent beads and mils, respectively. An odd 
number of tails appears only for the outcomes 117 and 77/77. Since the eight outcomes 
have equal probability, each of the four possible outcomes, given that F occurs, should 
also have an equal probability of 1/4. This suggests that we should assign the proba¬ 
bility of 2/4 - 1/2 to E, given that F occurs. This probability is called the conditional 
probability of E given F. 

In general, to find the conditional probability of E given 7, we use /•' as the sample 
space. For an outcome from E to occur, this outcome must also belong to E n F. With 
this motivation, we make the following definition. 


DEFINITION 2, Let E and F be events with pfF) > 0. The conditional proba¬ 
bility of E given F, denoted by p(E j F) ( is defined as 


P(E\F) = 


p(E n F) 
PiE) 
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EXAMPLE 3 


EXAMPLE 4 


EXAMPLE 5 


What is the probability that a bit string of length four, generated at random so that each 
of the 16 bit strings of length four is equally likely, contains at least two consecutive 
Os. given that its first bit is a 0? ( We assume that 0 bits and 1 bits are equally likely.) 


Solution: Let E be the event that a bit string of length four contains at least two consec¬ 
utive Os, and let F be the event that the first bit of a bit string of length four is a 0. The 
probability that a bit string of length four has at least two consecutive Os, given that its 
first bit is a 0. equals 


p(E | F) 


p{E n F) 

m 


Since EC\F ~ {0000,0001,0010,0011, 0100}, we see that p (E n F ) ~ 5/16. Since 
there are eight bit strings of length four that start with a 0, we have p (F) - 8/16 = 1/2. 
Consequently, 


p(E | F) 


5/16 

T/2 


5 

8 ‘ ■ 


What is the conditional probability that a family with two children has two boys, given 
they have at least one boy '.' Assume that each of the possibilities BB, BG, GB. and GG 
is equally likely, where B represents a boy and G represents a girl. 


Solution: Let E be the event that a family with two children has two boys, and let F be 
the event that a family with two children has at least one boy. It follows that E - {iff?}, 
F = {BB, BG, GB}, and E f! F = {BB}. Since the four possibilities are equally likely, 
it follows that plF) = 3/4 and p(E n F) - 1/4. We conclude that 


p(E n F) 1/4 
" P (F) ~~ 3/4 


1 

r ■ 


INDEPENDENCE 


Suppose a coin is flipped four times, as described in the i [Production to our discussion of 
conditional probability. Does knowing that the first flip comes up tails, (event F), alter 
the probability that tails comes up an odd number of times (event £)? In other words, 
is it the case that p(E ; F) - p(E)'! This equality is valid for the events E and F , since 
p(E | F) - 1/2 and p(E) = 1/2. Because this equality holds, we say that E and F are 
independent events. 

Since p(E \ F) - p{E O F)/p(F), asking whether p\E \ F) - p(E) is the same as 
asking whether p(E Pi r ) - p(E)p(F). This leads to the following definition, 

DEFINITION 3. The events E and F are said to be imkpendcnt if and only if 

p(F H F) = p{E)p{F). 


Suppose £ is the event that a randomly generated bit string of length four begins with 
a 1 and F is the event that a randomly generated bit string contains an even number of 
Os. .Are £ and F independent, if the 16 bit strings of length four are equally likely? 


PDF created with pdfFactory trial version www.pdffactorv.com 






272 


4 / Counting 



Solution: There are eight bit strings of length four that begin with a 1: 1000, 1001, 
1010.1011. 1100,1101,1110, and 1111. There are also eight bit strings of length four 
that contain an even number of Is; 0000, 0011, 0101, 0110, 1001, 1010, 1100, 111!. 
Since there are 16 bit strings of length four, il follows that 

p(E) = p(F) = 8/16 = 1/2, 

Because E 0 F = {U11,1100,101.0,1001}, we see that 

p(E n F) -4/16 = 1/4. 

Since 

P(EDF) - 1/4 = fl/2)(l/2) = p(E)p(F), 

we conclude that E and F are independent. ■ 

EXAMPLE 6 

Assume, as in Example 4, that each of the four ways a family can have two children is 
equally likely. Are the events £, that a family with two children has two boys, and F, 
that a family with two children has at least one boy, independent? 

Solution: Since E = {SB}, we have p(E) = 1/4. In Example 4 we showed that 
p(F) = 3/4 and that p(ECiF) = 1/4. Since p{EC\F) = 1/4^ 3/16 - (l/4)(3/4) - 
/>(£ )/>(£), e vents E and F are not independent. ■ 

EXAMPLE 7 

Are the events £, that a family with three children has children of both sexes, and £, 
that a family with three children has at most one boy, independent? Assume that the 
eight ways a family can have three children are equally likely. 

Solution: By assumption, each of the eight ways a family can have three children, 
BBB, B8G , BGB , BGG, CRB, GBG, GGB, and GGG, has a probability of 1/8. Since 
E = [BBC, BGB, BGG, GBB, GBG, GGB}, F *. {BGG, GBG, GGB, GGG}, and ED 
F ~ {BGG, GBG, GGB}, it follows that p{E) - 6/8 - 3/4 ,p(F) = 4/8 - 1/2, and 
P (E r F) = 3/8. Since 

p[E p(E)p(F), 

we conclude that £ and F are independent, (This conclusion may seem surprising. In¬ 
deed, if we change the number of children, the conclusion may no longer hold. See 
Exercise 19 at the end of this section.) ■ 

web 

BERNOULLI TRIALS AND THE BINOMIAL DISTRIBUTION 

Suppose that an experiment can have only two possible outcomes. For instance, when 
a bit is generated at random, the possible outcomes are 0 and 1. When a coin is flipped, 
the possible outcomes are- heads and tails. Each performance of an experiment with two 
possible outcomes is called a Bernoulli trial, after James Bernoulli, who made impor¬ 
tant contributions to probability theory. In general, a possible outcome of a Bernoulli 
trial js called a success or a failure. If p is the probability of a success and q is the 
probability of a failure, it follows that /> + <?= 1, 
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EXAMPLE 8 


THEOREM 1 


web 


Many problems cm be solved by determining the probability of k successes 
when an experiment consists of n independent Bernoulli trials. Consider the following 
example. 


A coin is biased so that the probability of heads is 2/3. What is the probability that 
exactly four heads come up when the coin is Hipped seven times, assuming that the 
flips are independent? 

Solution: There are 2 7 - 128 possible outcomes when a coin is flipped seven times. 
The number of ways four of the seven flips can be heads is C(7,4). Since the seven 
flips are independent, the probability of each of these outcomes is (2/3) 4 (l/3) 3 . Conse¬ 
quently, the probability that exactly four heads appear is 

C(7,4>(2/3) 4 (l/3) 3 = ^y 16 

560 

“ 2TS7 ' ■ 


Following the same reasoning as used in Example 8, we can establish the following 
theorem, which tells us the probability of k successes in n independent Bernoulli trials. 


PROBABILITY OF k SUCCESSES IN it INDEPENDENT BERNOULLI 
TRIALS Die probability of k successes in n independent Bernoulli trials, with 
probability of success p and probability of failure q = 1 - p, is 

C(n, k)p k <f~ k . 


Proof: When n Bernoulli trials are carried out, the outcome is an n-tuple (t\, u. l, s ). 

where r, - S (for success) or f, - F (for failure) for i = 1.2,_ n. Since the n trials 

are independent, fhe probability of each outcome of n trials consisting of k successes 
and n - k failures (in any order) is p i q”~ i , Since there are C(ri k) ^-tuples of Ss and 
Fs that contain k Ss, the probability of k successes is 

C(rt,k)p k ef \ □ 


.fames Bernoulli i 1654-1765). James Bernoulli (also known as Jacob I), was born in Basel, Switzerland. 
He is one of the eight prominent mathematicians in the Bernoulli family (see Section S. 1 for the Bernoulli 
family tree of mathematicians). Following his father's wish, tames studied theology and entered the min¬ 
istry. But contrary to the desires of his parents, he also studied mathematics and astronomy. He traveled 
throughout Europe from 1676 to 1682, learning about the latest discoveries in mathematics and the sci¬ 
ences. Upon returning to Basel in 1682. he founded a school for mathematics and the sciences. He was 
appointed professor of mathematics at the University of Basel in 1687, remaining in this position for the 
rest of his life. 

James Bernoulli is best known for the work Ars CrmctondU published 8 years after his death. In this 
work, he described the known results in probability theory and in enumeration, often providing alternative 
proofs of known results. Thts work also includes the application of probability theory to games of chance 
and his introduction of the theorem known as the law of large numbers. This law states that if e 7- 0, as n 
becomes arbitrarily large the probability approaches 1 that the number of times an event E occurs during n 
trials is within e Of p{E) 
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EXAMPLE 9 


EXAMPLE 10 


We denote by b{k\ n, p) the probability of k successes in n independent Bernoulli trials 
with probability of success p and probability of failure q ~ 1 - p. Considered as a 
function of k, we call this function the binomial distribution. Theorem ! tells us that 
b{k\ n, p) = C(n, k)p k q n - k . 


What is the probability that exactly eight 0 bits are generated when 10 bits are generated 
with the probability that a Obit is generated is 0.9, the probability that a 1 bit is generated 
is 0.1, and the bits are generated independently'’ 

Solution: By Theorem 1, the probability that exactly eight 0 bits are generated is 
b(8; 10,0.9) - Cf 10. 8)(0.9) s (0.1) 2 = 0.1937102445. ■ 


Note that the sum of the probabilities that there are k successes when n independent 
Bernoulli trials are carried out, for k = 0, 1, 2,.... ft, equals 

V C(n, k)p k q n ~ k = (p + q) n = 1, 

t-o 

as should be the case. The first equality in this string of equalities is a consequence of 
the binomial theorem. The second equality follows since q - 1 - p. 


RANDOM VARIABLES 


Many problems are concerned with a numerical value associated with the outcome of 
an experiment. For instance, we may want to know the probability that there are nine 
1 bits generated when 10 bits are randomly generated, or we may want to know the 
probability that a coin comes up tails 11 times when it is flipped 20 times. To study 
problems of this type we introduce the concept of a random variable. 


DEFINITION 4. A random variable is a function from the sample space of an 
experiment to (be set of teal numbers. That is, a random variable assigns a real num¬ 
ber to each possible outcome. 


Remark: Note that a random variable is a function. It is not a variable, and it is not 
random! 


Suppose that a coin is flipped three times. Let X(i) be the number of heads that appear 
when i is the outcome. Then the random variable X(t) takes on the following values: 

X(HHli) = 3 , 

X(HHT) - X(HTH) = X(THH) - 2, 

XiTTH) = X(THT) = X(HTT) = 1 , 

X(TTT) = 0 . ■ 
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EXAMPLE 1 it Lei X be the sum of the numbers that appear when a pair of dice is rolled. What are the 

values of this random variable for the 36 possible outcomes (/,/), where i and j are the 
numbers that appear on the first die and the second die, respectively, when these two 
dice are rolled? 

Solution: The random variable A takes on the following values; 

X({1, D) = 2, 

,Y((1,2)) = X((2,1)) - 3, 

*{(U)) = X((2,2)) = Ami)) = 4, 

A'((l,4}) = A (12,3)1 = X((3.2)) = A((4, 1)} = 5, 

A'((l,5)) = A{(2,4)) = A((3, 3)) - X((4. 2)) - XT(5. I)) - 6, 

A’((l, 6)) = A((2,5)) = A((3,4)) - X((4, 3)) - A'«5,2)> - AT(6,1)) = 7, 

A F ((2, 6)) - A'((3,5)) = A((4,4V) - X((5, 3)) - X((6,2» = 8, 

A'((3, 6)) = X((4,5» - Y!i5. L. = AT6 3}) - 9, 

A’((4, 6)) - A((5,5)) = X((6,4 t) = 10, 

X((5, 6)) = A((6,5)) =11. 

A((6, 6)) = 12. ■ 

EXPECTED VALUES 


Many questions cart be formulated in terms of the value we expect a random variable 
Wffr to take, or more precisely, the average value of a random variable when an experiment 
is performed a large number of times. Questions of this kind include : How r many heads 
are expected to appear when a coin is flipped 100 times? What is the expected number 
of comparisons used to find an element in a list using a linear search? To study such 
questions we introduce the concept of the expected value of a random variable. 

DEFINITION 5. The expected value (or expectation) of (he random variable X(j) 
on the sample space S is equal to 

e(x) = y^pimn 

teS 


Note when the sample space 5 has n elements S = {jr la ,r 2 . ,t„}, E(x) - 

p(x,)x(x,% 

Remark: We are concerned only with random variables w ith finite expected values here. 


EXAMPLE 12 A fair coin is flipped three times. Let S be the sample space of the eight possible out¬ 

comes, and let X be the random variable that assigns to an outcome the number of heads 
in this outcome. What is the expected value of X? 

Solution: Jn Example 10 we listed the values of X for the eight possible outcomes 
when a coin is flipped ihree tiroes. Since the coin is fair and the flips are independent, 
the probability of each outcome is 1/8. Consequently, 
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E(X) - + X(HHT) + X(HTH) + X(THH) + X(TTH) 

+ X(THT) + X(HTT) + X(TTT)) 

= ^C3 + 2 + 2 + 2 + J + I+ I+0) 

ft 

12 

” I 

_ 3 

~ 2' m 


When an experiment has relatively few outcomes, we can compute the expected 
value of a random variable directly from its definition, as was done in Example 12, 
However, when an experiment has a large number of outcomes, it may be inconvenient 
to compute the expec ted value of a random variable directly from its definition. Instead, 
we can find the expected value of a random variable by grouping together all outcomes 
assigned the same value by the random variable. In particular, suppose that X is a ran¬ 
dom variable with range A'i.V), and let p(X - r) be the probability that the random 
variable X takes the value r. Consequently, p(X — r) is the sum of the probabilities of 
the outcomes s such that X(s) = r. It follows that 

E(X) = V p{X = r)r, 

rfXtS) 

Examples 13 and 14 illustrate the use of this formula. In Example 13 we will find the 
expected value of the sum of the numbers that appear on two fair dice when they are 
rolled. In Example 14 we will find the expected value of the number of successes when 
n Bernoulli trials are performed. 


EXAMPLE 13 What is the expected value of the sum of the numbers that appear when a pair of fair 

dice is roiled? 

Solution: Let X be the random variable equal to the sum of the numbers that appear 
when a pair of dice is rolled. In Example 11 we listed the value of X for the 36 outcomes 
of this experiment. The range of X is {2,3,4,5,6,7,8,9,10, 11,12}. By Example 11 we 


see that 






P (X 

- 2) = 

p(X = 

12) 

= 1/36, 


P(X 

- 3) = 

P(X = 

11} = 

= 2/36 

= 1/18, 

p(X 

= 4) = 

P(X = 

10 ) = 

= 3/36 

= 1/12, 

p(X 

= 5) = 

P(X = 

9) = 

4/36 = 

1/9, 

piX 

= 6) = 

PiX = 

8) = 

5/36, 


P(X 

- 7) = 

6/36 - 

i/6. 



Substituting these 

: values 

in the formula, we have 
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EXAMPLE 14 


THEOREM 2 


£m = 2 .^ + 3 ^M.I + 5.I.6.1 + 7.I 


A + 9 ■ - + 10 ■ — 4 H ■ + 12 ■ - 1 

36 9 12 18 36 


What is the expected value of the number of successes when n Bernoulli trials are 
performed, where p is the probability of success on each trial? 

Solution: Let X be the random variable equal to the number of successes in /; trials. By 
Theorem 1 we see that p(X = k) = C(rt, k)p k q n ~ k . Hence, from the formula for the 
expected value of a random variable that groups together outcomes assigned the same 
value by the random variable, we have 

EiX) - V kp[X - k) 

tt-i 

= ^ kC(n, k)p k q n ~ k 

r-1 


= ],k-i)p k q a ~ t 

= npVcffl* l.c- li.T '(/ ‘ 

i-- 1 

= - [. : ; 

.! = 0 

- np(p + qf 1 
= ftp. 

The third equality follows from the fact that C(n, k ) - nC(n-l,fc- l }/Jfe, which follows 
from Exercise 33 of Section 4,3. The fifth equality is obtained by shifting the index of 
summation with j - k~ 1 so that j ranges from0 to n - 1 when k ranges from 1 to n. The 
sixth equality follows from the Binomial Theorem. The seventh equality follows since 
p + q = 1. From this computation, we conclude that the expected value of X equals np. 
This means that the expected number of successes in n Bernoulli trials is np. ■ 


Theorem 2 establishes some useful properties of expected values, including that 
the expected value of the sum of random variables is the sum of their expected values. 

If X and Y are random variables on a space S, then E(X 4 T) - E(X) + E{Y). Fur¬ 
thermore, if Xi, i * 1,2,..., n with n a positive integer, are random variables on S, 

then E(X) = E{X\) + J?{Xj}) 4-F E(X n ). Moreover, ifrt and b are real numbers, 

then E(aX 4 b) = aE(X) 4 b. 
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EXAMPLE 15 


EXAMPLE 16 


Proof: The first result follows directly from the definition of expected value, since 

E{X+Y) = ^ p(.vii A'fv) + fti)) 

.lELS 

= V pis) X(s) + P(-y)Tfi) 
ms ,te s 

= E(X) 4 £(.}’). 

r rhe ease with n random variables follows easily using mathematical induction 
from the case of two random variables. Finally, note that E(aX + b) - v _. , pin 
(aX(s) 4- b) - aP(s)X{x) + bzL se s pW - aE(X) + b since p( S ) = 1, □ 

Theorem 2 can be useful for computing expected values, since many random vari¬ 
ables are sums of simpler random variables, as Examples 15 and 16 illustrate. 


Use Theorem 2 to find the expected value of the sum of the numbers that appear when 
a pair of fair dice is rolled. (This was done in Example 13 without the benefit of this 
theorem.) 

Solution: Let X] and X 2 be the random variables with Xj((r, j)) = i and XjUi. j)) = /, 
so that A'i is the number appearing on the first die and Xj is the number appearing on 
the second die. It is easy to see that E{X } ) = E{Xt) = 7/2 since both equal (1+2- 
3 + 4 + 5 + 6)/6 = 21/6 - 7/2. Tlie sum of the two numbers that appear when (he two 
dice are rolled is the sum X| + X>, By Theorem 2, the expected value of the sum is 
E(X i - X 2 ) = E(X } ) + E(X 2 ) = 7/2 + 7/2 - 7. ■ 


In Example 14, the expected value of the number of successes when n Bernoulli trials 
are performed, where p is the probability of success on each trial, was shown by direct 
computation to be np. Show how this result can be found using Theorem 2. 

Solution: Let X; be the random variable with Xj((f[, (3 .f, 1 )) = 1 if f, is a success 

and A7((f|, t 2 .(„)) = 0 if r, is a failure. The expected value of X, is E(X,) ~ l ■ p + 

0 ■ ( ! - p) = p for i = 1, 2,..., n. Let X = X 1 + X 2 i -1- X„ so that X counts the num¬ 

ber of successes when these n Bernoulli trials are performed. Theorem 2, applied to the 
sum of n random variables, shows that £(X) = FAXE + EtXi) + —f E(X n ) - np M 

We have already discussed independent events. We will now define what it means 
for two random variables tn be independent. 


INDEPENDENT RANDOM VARIABLES 


DEFINITION 6 The random variables X and T on a sample space S are indepen¬ 
dent if 

p(X(s), ~ n and Y{$) = r 2 ) = p(X(s) = n) • p(Y(s) = r 2 ), 

or in words, if the probabihty that X{s) ~ r t and TO) = r 2 equals the product of 
the probabilities that X(s) - r x and Y(s) -- n, for all real numbers rj and r h 
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EXAMPLE 17 

Are the random variables A) and X'? from Example 15 independent? 

Solution; Lei 5 = {1,2,3, 4,5,6}, and let i £ 5 and j £ 5. Since there are 36 possible 
outcomes when the pair of dice is rolled and each is equally likely, we have 

p(X i - i and Xi - j) - 1/36. 

Furthermore. p\X\ = i) - 1/6 and p(X 7 - j) = 1/6, since the probability that i ap¬ 
pears on the first die and the probability that j appears on the second die are both 1/6. 
It follows that 

p(X j ■ t andXj - j) -- 1/36 = (1/61(1/6) = p{X\ = i)p(X ? = jl 

so X\ and X 2 are independent. ■ 

EXAMPLE 18 

Show that the random variables X\ and X = Aj + X;, where X’i and Xi are as defined 
in Example 15, are not independent. 

Solution: Note that p{X\ = 1 and X = 12) - 0, since X| = 1 means the number 
appearing on the first die is 1, which implies that the sum of the numbers appearing on 
the two dice cannot equal 12. On the other hand, p(X\ = 1) = 1/6 arid p(X = 12) = 
1/36. Hence p{X\ = 1 and X' = l2)¥=p(X\ = Ij-piX = 12). This counterexample 
shows that Xj and X are not independent. ■ 

The expected value of the product of two independent random variables is the prod¬ 
uct of their expected values, as Theorem 3 shows. 

THEOREM 3 

If X and Y are independent random variables on a space S, then E(XY) ■<= E(X)E(Y). 

Proof: From the definition of expected value and since X and Y are independent ran¬ 
dom variables, it follows that 

F(XY) ~ V Xis)Yh)p(s) 

its 

= X ' I'd 1 = > \ and F(r) - r 2 ) 

- X fj'r 0 = ri) • p(Y(s) = n) 

texisu jemi 

- f ^ r t p{X (r) = >’,))■! V np(YiX) ^ r 2 )j 

r,em) !|A 

- FiX'.Fi ) 

This completes the proof. □ 

web 

VARIANCE 

The expected value of a random variable tells us its average value but nothing about how 
widely its values are distributed. For example, if AT and Fare the random variables on the 
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THEOREM 4 


EXAMPLE 19 


EXAMPLE 20 


set 5 = {1,2,3,4,5,6}, with X(a) - 0 for all ,r e S and F(a’) = -1 if s G. {1,2,3} and 
>'•■ ' ) — 1 if s G {A, 5,6}, then the expected values of ,Y and Y are both zero. However, 
the random variable X never varies from 0, while the random variable Y always differs 
from 0 by 1. The variance of a random variable helps us characterize how widely a 
random variable is distributed. 

DEFINITION 7 Let X be a random variable on a sample space & The variance 
of X, denoted by V(X), is 

V'iX) - Y(X(*) - EiX)?m 

s££ 

The standard deviation of X, tooted' trfX), is defined toTw y'VfXj. 


The following theorem provides a useful simple expression for the variance of a 
random variable. 


If X is a random variable on a sample space 5, then V(X) - E(X 2 } ™ E{Xf. 

Proof: Note that 

V(X) - Y:X; A , - Emfp(s) 

sES 

- v v i - 2 EiX ) Y ,YH• + E\XY Y p(s) 

•eS fzs 7s s 

= Ei.V i - 2E(X)E{X) + E(xf 

- EfX 2 ) - £(X) 2 

We have used the fact that S p v. - 1 in the next-to-last step. □ 

What is the variance of the random variable X with Aft) = [ if a Bernoulli trial is a 
success and Xit ) = 0 if it is a failure, where p is the probability of success? 

Solution: Since X takes only the values 0 and t. it follows that Arif ) = Xu). Hence, 
V(Xl =■ £|Xfo -■ EIX) 1 -■ p- p 2 -- pi. 1 - p) - pq. g 


What is the variance of the random variable X(ii, j)) - 2i. where i is the number ap¬ 
pearing on the first die and j is the number appearing on the second die, when two dice 
are rolled? 

Solution: We will use Theorem 4 to find the variance of X, To do so, we need to find the 
expected values of X and A' 2 , Note that since p(X - Am is 1/6 for k - 2,4 6.8, 10, 12 
and is 0 otherwise, 

EiX = (2+4 + 6 + 84 10 - 12J/6 - 7, 
and 

EiX 1 ) - (2 2 + 4 2 4 6 2 + 8 2 + 10' 4 12 2 )/6 - 182/3. 
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THEOREM 5 


EXAMPLE 21 


EXAMPLE 22 


It follows from Theorem 4 that 

V{X) = £(A' 2 ) - E(X) 2 = 182/3 - 49 = 45/3. ■ 


Another useful fact about variances is that the variance of the sum of two indepen¬ 
dent random variables is the sum of their variances. This result is useful for computing 
the variance of the result of n independent Bernoulli trials, for instance. 

ffJagKf J" are tw kdqicndeiR randena vaiafetenitt'f sat$gi&®piee$, itoi VTA + 
¥) » V(X) + VQfoJMmnaim, i£ Xt>i ~ n apfesttve integer, 

are pairwiee mdependerrt random variables on S, then P<J?i + 4 X m i = 

V(Xi) + V(X 2 ) + ~- + V(X R ), 

Proof: From Theorem 4, we have 

V(X + Y) - E((X + F) 2 ) - EiX + Y f. 

It follows that 

V(X + Y) = E(X 2 + 2 XY + Y 2 ) - (E(X) + E(Y)) 2 

- EiX 1 ) + 2E(XY) + E{Y 2 ) - EiXr • 2£(X)£(F) - E(F) 2 . 

Since X and ¥ are independent, by Theorem 3 we have E(XY) = E(X)E(Y). It follow s 
that 


VCX + Y) = (E(X 2 ) - FAX) 2 ) + (E(Y 2 ) - E(Y) 2 ) 

= Vi A , -r Vi 1 i. 

The case with n pairwise independent random variables can be proved using mathe¬ 
matical induction; the proof is left to the reader. □ 


Find the variance and standard deviation of the random variable X whose value when 
two dice are rolled is X((i, />J = i + j, where i is the number appearing on the first die 
and j is the number appearing on the second die. 

Solution: l ei X\ and Xi be the random variables defined by A'[((i, j)) - i and 
AA(( /, j)) - j for a roll of the dice. Then X - AT 4 X 2 and AT and X 2 are independent, 
as Example 17 showed. From Theorem 5 it follows that V(X) = V(A',) + V(AY). A 
simple computation as in Example 20 together with Exercise 43 in the Supplemen¬ 
tary Exercise at the end of the chapter tell us that VfAA) = V(AV) = 35/f2. Hence. 
V(X) = 35/12 4 35/12 - 35/6 and olX) = /35/6. ■ 


We will now find the variance of the random variable that counts the number of 
successes when n independent Bernoulli trials are carried out. 


What is the variance of the number of successes when n independent Bernoulli trials 
are performed, where p is the probability of success on each trial? 
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THEOREM 6 


m,b 


Solution: Let A, be the random variable with X,((ii, h .<«)) = 1 if U is a success 

and Xf((ti ,h _fj) = 0 if t< is a failure. Let X = A] + X 2 + ■ ■ ■ + X„. Then X counts 

the number of successes in the/! trials, from Theorem 5 it follows that V'tA) -- V(A]) + 
V(X 2 ) + ■ ■ ■ + V{X B ). Using Example 19 wc have that L(A, ) = pq for 1 = 1,2,..,. n. 
It follows that K(A) = npq, ■ 

CHEBYSHEV’S INEQUALITY 


How likely is it that a random variable takes a value far from its expected value? The 
following theorem, called Chebyshev’s inequality, helps answer this question by pro¬ 
viding an upper bound on the probability that the value of a random variable differs 
from the expected value of the random variable by more than a specified amount. 

Chebyshev’s Inequality Let A he a random variable on .a sample space S with 
probability function/?. If r Ls a positive real number, then 

- E(X)\ > r) < VQO/r 2 . 


Proof: Let A be the event 

A = {j £ S | |A(s) - E(X}\ a? r}. 

What we want to prove is that pi A) V(X)/r. Note that 

V(X) = V.AV; - E(X)) 2 p(s) 

s£S 

- ytXCv) - E(X)rp{s) + yiA'I.vl - EiXyr. 

sea i€A 

The second sum in this expression is nonnegative, since each of its summands is non¬ 
negative. Also, since for each element s in A, (X(s) - E(X)f > r 2 , the first sum in 
this expression is at least 7 Jf .. rr'.s:. Hence, V(.r) a f 2 p(s) - t 2 p(A), This 
is what we needed to prove. □ 

Chebyshev’s inequality, although applicable to any random variable, often fails to 
provide a practical estimate for the probability that the value of random variable exceeds 
its mean by a large amount. This is illustrated by the following example. 


Pafnuty Lvovich Chebyshev ( 1821 - 1894 ). Chebyshev was bum into the gentry in Okatovo, Russia. His 
father was a retired army officer who fought against Napoleon. In 1832 the family, with its nine children, 
moved to Moscow, where pafnuty completed his high school education at home. He entered the Department 
of Physics and Mathematics at Moscow University. As a Student, he developed a new method for approxi¬ 
mating the roots of equations. He graduated from Moscow University in 184] with a degree in mathematics, 
and he continued his studies, passing his master's exam in 1843 and completing his master’s thesis in 1846, 
Chebyshev was appointed in 1847 to a position as an assistant at the University of St. Petersburg. He 
wrote and defended a thesis in 1847. He became a professoral St, Petersburg in I860, a position he held until 
1882 His book on the theory of congruences written in 1849 was influential in the development of number 
theory His work on the distribution of prime numbers w as seminal. He proved Bertrand’s conjecture that 
for every integer n > 3, there is a prime between n and In - 2 Chebyshev helped develop ideas that were 
later used to prove the prime number theorem. Chebyshev’s work on the approximation of functions using 
polynomials is used extunsi vely when computers arc used to find values of functions. Chebyshev was also 
interested in mechanics. He studied the conversion of rotary motion into rectilinear motion by mechanical 
coupling. The Chebyshev parallel motion is three linked bars approximating rectilinear motion. 
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EXAMPLE 23 


EXAMPLE 24 


Let X be the random variable whose value is the number appearing when a fair die 
is rolled. We have E(X) = 7/2 (see Example 15} and V(A') = 35/12 (see Example 
20). Since the only possible values of X are 1, 2, 3, 4, 5, and 6, A' cannot take a value 
more than 5/2 from its mean, E(X) = 7/2. Hence, p(|X - ?/2j s r) = 0 if r > 5/2. By 
Chebyshev’s inequality we know that p(\X - V2\ r) < (35/12)/r 2 . For example, 
when r - 3, Chebyshev’s inequality tells us that p(|X - 7/21 ^ 3) ^ (35/12)/9 = 
35/108. which is a poor estimate, since p\\X - 7/2| > 3) - 0. ■ 


AVERAGE-CASE COMPUTATIONAL COMPLEXITY 

Computing the average-case computational complexity of an algorithm can be inter¬ 
preted as computing the expected value of a random variable. Let the sample space 

of an experiment be the set of possible inputs a Jt j = 1,2__ n, and let the random 

variable X assign to aj the number of operations used by the algorithm when given tty 
as input. Based on our knowledge of the input, we assign a probability p(aj) to each 
possible input value aj. Then, the average-case complexity of the algorithm is 

n 

E(X) - y^p{a J )X(a J ), 

f= 1 

This is the expected value of A'. 

In Example 24 we will illustrate how to find the average-case computational com¬ 
plexity of the linear search algorithm under different assumptions concerning the prob¬ 
ability that the element for which we search is an element of the list. 


Average-Case Computational Complexity of the Linear Search Algorithm We are 
given an dement x and a list of n distinct real numbers. The linear search algorithm, 
described in Section 2.1, locates x by successively comparing it to each element in the 
list, terminating when x is located or when all the dements have been examined and 
it has been determined that x is not in the list. What is the average-case computational 
complexity of (he linear search algorithm if the probability (fiat x is in the list is p and 
it is equally likely that x is any of the n elements in the list? (There are n T 1 possible 
types of input: the n numbers in the list and a number not in the list, which we treat as 
a single input.) 

Solution: In Example 4 of Section 2.2 we showed that 2f + 1 comparisons are used 
if x equals the ith element of the list and, in Example 2 of Section 2.2, that 2n + 2 
comparisons arc used if x is not in the list. The probability that x equals a;, the ith 
element in the list, is pin, and the probability that x is not in the list is q = 1 - p. 
It follows that the average-case computational complexify of the linear search algo¬ 
rithm is 

E - 3 pin + 5 pin + - ■ + (2n + 1 )phi + (In + 2)q 
- ^(3 + 5 + - ■ ■ + (In + 1)) + (2;j + 2 )q 

= Jd (n + l) a - l) +(2 b + 2)? 

~ pin + 2) + (In + 2)q. 
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(The third equality follows from Example 2 of Section 3,2.) For instance, when r is 
guaranteed to be in the list, we have p = 1 (so that the probability that x = <% is 1 in 
for each i) and q = 0, Then E ~ n + 2, as we showed in Example 4 in Section 2.2, 
When p , the probability that a is in the list, is 1/2, it follows that q — 1 - p # 1/2, 
so that £ - (n + 2)/2 + n+l = (3n+4)/2. Similarly, if the probability that x is in the list 
is .3/4, we have p - 3/4 and q - 1/4, so that E = 3(n + 2)/4+ (n + 1 )/2 = (5n + 8)/4 
Finally, when x is guaranteed not to be in the list, we have p = 0 and q = 1. It fol¬ 
lows that F. = 2n + 2, which is not surprising since we have to search the entire list. ■ 


Exercises 


1. What probability should be assigned to the outcome of 
heads when a biased coin is tossed, if heads is three 
times as likely to come up as tails'? What probability 
should be assigned to the outcome of tails'? 

2. Find the probability of each outcome when a loaded die 
is rolled, if a 3 is twice as likely to appear as each of 
the other five numbers on the die, 

3. Find the probability of each outcome when a biased die 
is rolled, if rolling a 2 or rolling a 4 is three times as 
likely as rolling each of the other four numbers on the 
die and it is equally likely to roll a 2 or a 4, 

4. Show that conditions (i) and (ii) are met under 
Laplace's definition of probability, when outcomes 
arc equally likely. 

5. A pair of dice is loaded. The probability that a 4 ap¬ 
pears on the first die is 2/7, and rhe probability that a 
1 appears on the second die is 2/7. Other outcomes for 
each die appear with probability 1/7, What is the prob¬ 
ability of 7 appearing as the sum of the numbers when 
the two dice are rolled? 

6. Suppose that F and F are events such that p(E) - 0.8 
and p(F) = 0.6. Show' that p(E n F) > 0,4. 

7. Show that if F. and F are events, then p(E n F) 2 
p\E) + p(F t l, This is known as Bemferroni’s in¬ 
equality. 

8. Ike mathematical induction to prove the following 
generalization of Bonfemmi’s inequality: 

p(/fi riEjil tie,,) 

- P(F 1 ) * p(E 2 ) + ■■■ + p(F„) - in - 1), 
where E 1 . £3. E„ are n events, 

9. Shttw that ifEi.Ej_ E„ are events from a finite sam¬ 

ple space.then 

plFi IJ Lj ■ ■ ■ U E„) 

p(E j) + p[Ei) + ■■■ + p(Ej. 

T his is known as Boole’s inequality. 

10. Show that if E and F arc independent events, then E 
and F are also independent events. 


11, If E and F are independent events, prove or disprove 
that E and F are necessarily independent events. 

Exercises 12-44 involve the probability that at least two 
people in a group have the same birthday. 

12, What is the probability that two people have the same 
birthday ?For this problem assume that all the 366dates 
are equally likely as birthdays. 

*13. a) What is the probability that in a group of n people, 
there are at least two with the same birthday'? For 
this problem assume that all of the 366 dates are 
equally likely as birthdays. (Hint: Find the proba¬ 
bility that in a group of n people, the birthdays of 
all the people are different.) 
b) How many people are needed to make the probabil¬ 
ity greater than 1 12 that at least two people have the 
same birthday? 

* 14 . February' 29 occurs only in leap years. Years divisible 
by 4, but not by 100 are always leap years. Years di¬ 
visible by 100, but not by 400, are not leap years, but 
years divisible by 400 are leap years. 

a) What probability distribution for birthdays should 
be used to reflect how often February 29 occurs? 

b) Answer the question asked in part (a) of Exercise 
13 using this probability distribution. 

15. What is the conditional probability that exactly four 
heads appear when a fair coin is flipped five times, 
given that the first flip came up heads? 

16 . What is the conditional probability that exactly four 
heads appear when a fair coin is flipped five times, 
given that the first flip came up tails? 

17. What is the conditional probability that a randomly 
generated bit string of length four contains at least two 
consecutive Os, given that the first bit is a 1? (Assume 
the probabilities of a 0 and a 1 are the same. ) 

18. Let E he the event that a randomly generated hit string 
of length three contains an odd number of Is, and let F 
be the event that the string starts with I. Are E and F 
independent? 
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19. Let E and F be the events that a family of n children 
has children of both sexes and has at most one hoy, re¬ 
spectively, Are E and F independent if 

a) n = 2? b) n = 4'.' 

C) n ■- 5? 

20. Assume that the probability a child is a boy is 0,51 and 
that the sexes of children bom into a family are inde¬ 
pendent, What is the probability that a family of live 
children has 

a) exactly three boys? 

b) at least one boy ? 

c) at least one girl? 

d) all children of the same sex.? 

21. A group »f si x people play the game of "odd person out" 
to determine who will buy refreshments. Each person 
flips a fair coin, If there is a person whose outcome is 
not the same as that of any other member of the group, 
this person has to buy the refreshments. What is the 
probabilit y that there is an odd person out after the coins 
are hipped once ,' 

22. Find the probability that a randomly generated bit 
string of length 10 does not contain a 0 if bits are inde¬ 
pendent and if 

a) a 0 bit and a t bit are equally likely, 

b) the probability that a bit is a 1 is 0.6. 

c) the probability that the ith bit is a 1 is 1/2' for i ~ 

1,2,3.I O’ 

23. Find the probability that a family with live children 
does not have a boy, if the sexes of children are inde¬ 
pendent and if 

a) a boy and a girl are equally likely. 

b) the probability of a boy is 0.51. 

cT the probability that the ith child is a boy is 0.51 - 
li/100). 

24. Kind the probability that a randomly generated bit 
string of length 10 begins with a 1 or ends with a (X) 
for the same conditions as in parts (a), (b), and (c) of 
Exercise 22. if bits arc generated independently. 

25. Find the probability that the firs t child of a family with 
five children is a boy or that the last two children of the 
family are girls, for the same conditions as in parts (a), 
(hi, and (c) of Exercise 23- 

26. Find each of the following probabilities when n inde¬ 
pendent Bernoulli trials are carried out with probability 

of Success p, 

a) the probability of no successes 

b) the probability of at least one success 
cl the probability of at most one success 

d) the probability of at least two successes 

27. Find each of the following probabilities when n inde¬ 
pendent Bernoulli trials are carried out w ith probability 
of success p. 

a) the probability of no failures 
h) the probability of at least one failure 


c) the probability' of at most one failure 

d) the probability of at least two failures 

28. What is the expected number of heads that come up 
when a fair coin is flipped 10 times? 

29. What is the expected numberof timesa 6 appears when 
a fair die is rolled 10 times? 

30. A coin is biased so that the probability a head comes up 
when it is flipped is 0.6. What is the expected number 
of heads that come up when it is flipped 10 times? 

31. What is the expected sum of the numbers that appear 
on two dice, each biased so that a 3 comes up twice as 
often as each other number? 

32. What is the expected value when a $1 lottery ticket is 
bought in which, the purchaser wins exactly $10 mil¬ 
lion if the ticket contains the six wanning numbers cho¬ 
sen from the set {1, 2, 3,..., 50} and the purchaser wins 
nothing otherwise? 

33. The final exam of a discrete mathematics course con¬ 
sists of 50 true/false questions, each worth two points, 
and 25 multiple-choice questions, each worth four 
points. The probability that Linda answers a tme/false 
question correctly is 0.9, and the probability that she 
answers a multiple-choice question correctly is 0.8. 
What is her expected score on the final? 

34. What is the expected sum of the numbers that appear 
when three fair dice are rolled? 

35. Suppose that the probability that x is in a list of n dis¬ 
tinct integers is 2/3 and that it is equally likely that .r 
equals any element in the list, Find the average number 
of comparisons used by the linear search algorithm to 
find x or to determine that it is not in the list. 

*36, Suppose the probability that x is the ith element in a list 
of n distinct integers is + 1)]. Find the average 
number of comparisons used by the linear search algo¬ 
rithm to find x or to determine that it is not in the list. 

In this section we have studied experiments with finitely 

many outcomes. In Exercises 37-40 we study an experi¬ 
ment with countably many outcomes. A coin is flipped un¬ 
til it comes up tails. The sample space of the experiment is 

{T, HT, HHT „ HHHT, HHHHT, . 

The probability the coin comes up tails is p. 

37. What is the probability that the experiment ends after 
n flips, that is, (he outcome consists of n - I heads and 
a tail? 

38. Show' that the sum of die probabilities of the possible 
outcomes equals 1. 

39. What is the probability that at most n flips are required 
for the experiment to end? 

40. What is the expected value of the number of flips re¬ 
quired for the experiment to end? 

41. At a party n people toss their hats into a pile in a closet. 
The hats are mixed up, and each person selects one at 
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random, What is the expected number of people who 
select their own hats? 

42, Let X(s') be a random variable, where X{/) is a nonneg¬ 
ative integer for all s S S, and let At be the event that 
X(,r) '• k, Show that FAX) = £'_j p(A k ). 

43, What is the variance of the number of heads that come 
up when a fair coin is flipped 10 times'? 

44. What is the variance of the number of times a 6 appears 
when a fair die is rolled 10 times? 

45. Let ,Y„ be the random variable that counts the differ¬ 
ence in the number of tails and the number of heads 
when n coins are flipped. 

a) What is the expected value of X„‘! 

b) What is the variance of ,Y„ 7 

4b. Provide an example that shows that the variance of the 
sum of two random variables is not necessarily equal to 
the sum of their variances when the random variables 
are not independent. 

47. Let A’ be a random variable on a sample space 5 such 
that X(.r) a 0 for alls E 5, Show that p{X{s) £ a) 
F.(X]!a for every positive real number a. This inequal¬ 
ity is called Markov’s inequality. 

4h, Suppose that the number of cans of soda pop filled in 
a day at a bottling piant is a random variable with an 
expected value of 10,00ft and a variance of 1000. 
a) Use Markov’s inequality (Exercise 47} to obtain 
an upper bound on the probability that the plant 


will fill more than 11,000 cans on a particular 
day. 

b) Use Chebyshev’s inequality to obtain slower bound 
on the probability that the plant will fill between 
9000 and 11,000 cans on a particular day. 

49. Suppose that the number of tin cans recycled in a day 
at a recycling center is a random variable with an ex¬ 
pected value of 50,000 and a variance of 2500. 

a) Use Markov’s inequality (Exercise 47) to find an 
upper bound on the probability that the center will 
recycle more than 55,000 cans on a particular day. 

b) Use Chebyshev’s inequality to provide a lower 
bound on the probability that the center will recycle 
40,000 to 60,000 cans on a certain day. 

The covariance of two random variables X and Tors a sam¬ 
ple space S, denoted by CoviX, Y), is defined to be the ex¬ 
pected value of the random variable (X - B(X))(Y - E( f )). 

That is, Cov(X, Y) = E((X - E(X))(Y - EiY))). 

50. Show that Cov(X, Y) = EiXY) - E(X)E(Y), and use 
this result to conclude that CovlX T) = 0 if X and Y 
are independent random variables. 

51. Show that V'(.Y + Y) = V'lY) + V(Y) + 2 Cov(Y, Y), 

52. Find CovfX Tl if X and Y are the random variables 
with X((J, j)) = 2t and Y((i, j}) = / + j, where i and 
j are the numbers that appear on the first and second of 
two dice when two fair dice are rolled, 


4.6 _ 

Generalized Permutations and Combinations 

INTRODUCTION 

In many counting problems, elements may he used repeatedly. For instance, a letter or 
digit may be used more titan once on a license plate. When a dozen donuts are selected, 
each variety can be chosen repeatedly. This contrasts with the counting problems dis¬ 
cussed earlier in the chapter where we only considered permutations and combinations 
in which each item could be used at most once. In this section we will show how to 
solve counting problems where elements may be used more than once. 

Also, some counting problems involve indistinguishable elements. For instance, to 
count the number of ways the letters of the word SUCCESS can be rearranged, the place¬ 
ment of identical letters must be considered. This contrasts with the counting problems 
discussed earlier where all dements were considered distinguishable. In this section 
we will describe how to solve counting problems in which some dements are indistin¬ 
guishable. 

Moreover, in this section we will explain how to solve another important class of 
counting problem, problems involving counting the ways to place distinguishable el¬ 
ements in boxes. An example of this type of problem is the number of different ways 
poker hands can be dealt to four players. 
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EXAMPLE 1 


EXAMPLE 2 


THEOREM 1 


EXAMPLE 3 


Taken together, the methods described earlier in this chapter and the methods in¬ 
troduced in this section form a useful toolbox for solving a wide range of counting 
problems. When the additional methods discussed in Chapter 5 are added to this arse¬ 
nal, you will be able to solve a large percentage of the counting problems that arise in 
a wide range of areas of study. 


PERMUTATIONS WITH REPETITION 

Consider the following example of a counting problem when repetition is allowed. 


How many strings of length n can be formed from the English alphabet? 

Solution: By the product rule, since there are 26 Setters, and since each letter can be 
used repeatedly, we see that there are 26 fl strings of length n, ■ 


The following question involving probability also involves permutations with 
repetition. 


What is the probability of drawing three red balls in a row from an urn containing five 
red balls and seven blue balls if a ball is put back into the urn after it is drawn? 

Solution: By the product rule the number of successful outcomes—that is, the number 
of ways to draw* three red balls—is 5 3 , since for each drawing there are five red balls in 
the um. The total number of outcomes is 12 3 , since for each drawing there are 12 balls 
in the um. Thus, the desired probability is 5 3 /12 3 - 125/1728, This is an example of 

sampling with replacement, ■ 


The number of /--permutations of a set with n elements when repetition is allowed 
is given in the following theorem. 


The number of /"-permutations of a set of n objects with repetition allowed is n r . 

Proof: There are n ways to select an element of the set for each of the r positions in 
the /--permutation when repetition is allowed, since for each choice all n objects are 
available. Hence, by the product rule (here are n r r -permutations when repetition is 
allowed. □ 


COMBINATIONS WITH REPETITION 

Consider the following examples of combinations with repetition of dements allowed. 


How many ways are there to select four pieces of fruit from a bowl containing apples, 
oranges, and pears if the order in which the pieces are selected does not matter, only 
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EXAMPLE 4 


the type of fruit and not the individual piece matters, and there are at least four pieces 
of each type of fruit in the bowl? 


Solution: To solve this problem we list all the ways possible to select the fruit. There 
are 15 ways: 


4 apples 

3 apples, 1 orange 
3 oranges, 1 pear 
2 apples, 2 oranges 
2 apples, 1 orange, 1 pear 


4 oranges 
3 apples, 1 pear 
3 pears, 1 apple 
2 apples, 2 pears 
2 oranges, 1 apple, 1 pear 


4 pears 

3 oranges, 1 apple 
3 pears, 1 orange 
2 oranges, 2 pears 
2 pears, 1 apple, 1 orange 


The solution is the number of 4-combinations with repetition allowed from a three- 
element set, {apple, orange, pear}. | 


To solve more complex counting problems of this type, we need a general method for 
counting the r-combinations of an n-element set. In Example 4 we will illustrate such 
a method. 


How many ways are there to select five bills from a money bag containing $1 bills, 
$2 bills, $5 bills, $10 bills, $20 bills, $50 bills, and $100 bills? Assume that the order 
in which the bills are chosen does not matter, that the bills of each denomination are 
indistinguishable, and that there are at feast five bills of enph type. 

Solution: Since the order in which the bills are selected does not matter and seven dif¬ 
ferent types of bills can be selected as many as five times, this problem involves count¬ 
ing 5-combinations with repetition allowed from a set with seven elements. Listing all 
possibilities would be tedious, since there are a large number of solutions. Instead, we 
will illustrate the use of a technique for counting combinations with repetition allowed. 

Suppose that a cash box has seven compartments, one to hold each type of bill, as 
illustrated in Figure 1. These bins are separated by six dividers, as shown in the picture. 
The choice of five bills corresponds to placing five markers in the compartments holding 
different types of bills. Figure 2 illustrates this correspondence for three different ways 
to select five bills, where the six dividers are represented by bars and the live bills by 
stars. 

The number of ways to select five bills corresponds to the number of ways to ar¬ 
range six bars and five stars. Consequently, the number of ways to select the five bills is 
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I KH RK 2 Examples of Ways to Select Five Bilis. 


the number of ways to select the positions of the live stars, from 11 possible positions. 
This corresponds to (he number of imordered selections of 5 Objects from a set of ] I 
objects, which can he done in Cl ! 1, 5) ways. Consequently, there arc 

CM 1.5) - ~~ - 462 

3! ft! 

ways to choose five hills from the bag with seven types of bills, ■ 


The following theorem generalizes this discussion. 


There are C(n +■ r - 1, r) r-cointimations from a set with n elements when repetition 
of dements is allowed. 

hoof: Each (--combination of a set with n elements when repetition is allowed can be 
represented by a list of n 1 bars and r stars. The n - 1 bars are used to mark off n 
different cells, with the fth cell containing a star for each time the r'th dement of the set 
occurs in the combination. For instance, a 6-combination of a set with four dements is 
represented with three bars and six stars. Here 


represents the combination containing exactly two of the first element, one of the second 
dement, none of the third element, and three of the fourth element of the set. 

As we have seen, each different list containing n ■ I bars and r stars corresponds 
to an (--combination of the set with « dements, when repetition is allowed, '['he num¬ 
ber of such lists is On - I + >\ r), since each list corresponds to a choice of the r 


PDF created with pdfFactory trial version www.pdffactorv.com 



















290 4 / Counting 


EXAMPLE 5 


EXAMPLE 6 


positions to place the r stars from the n — 1 + r positions that contain r stars and n - 1 
bars. □ 

The following examples show how Theorem 2 is applied. 


Suppose that a cookie shop has four different kinds of cookies. How many different 
ways can six cookies be chosen? Assume that only the type of cookie, and not the 
individual cookies or the order in which they are chosen, matters. 

Solution: The number of ways to choose six cookies is the number of 6-combinations 
of a set with four elements. From Theorem 2 this equals C(4 + 6 - 1,6) = 09,6). 
Since 

9-87 

09 to = 09,3) = — - - 84, 

there are 84 different ways to choose the six cookies. ■ 


Theorem 2 can also be used to find the number of solutions of certain linear equa¬ 
tions where the variables are integers subject to constraints. This is illustrated by the 
following example. 


How man}' .solutions does the equation 

A'l + X2 + Jfj — 11 

have, where jq, xi, and x 3 are nonnegative integers? 

Solution: To count the number of solutions, we note that a solution corresponds to a 
way of selecting 11 items from a set with three elements, so that .t} items of type one. 
x 2 items of type two, and items of lype three are chosen. Hence, the number of 
solutions is equal to the number of 11 -combinations with repetition allowed from a set 
with three elements. From Theorem 2 it follows that there are 

C(3+ 1( - 1,11} = Cd3.ll) - C(13, 2) = = 78 

solutions. 

The number of solutions of this equation can also be found when the variables 
are subject to constraints. For instance, we can find the number of solutions where the 
variables are integers with x, =: 1, x 2 s 2, and x 3 -- 3. A solution to the equation 
subject to these constraints corresponds to a selection of 11 items with xq items of type 
one, X 2 items of type two, and x 3 items of type three, where, in addition, there is at 
least one item of type one, two items of type two, and three items of type three. So, 
choose one item of type one. two of type two, and three of type three. Then select five 
additional items. By Theorem 2 this can be done in 

C(3 + 5 - 1,5) = <7(7.5) = CO, 2) = = 21 

ways. Thus, there are 21 solutions of the equation subject to the given constraints. ■ 
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EXAMPLE 7 


The following example shows how counting the number of combinations with rep¬ 
etition allowed arises in determining the value of a variable that is incremented each 
lime a certain type of nested loop is traversed. 


What is the value of k after the following pseudocode has been executed? 


k := 0 

for t'j : - 1 to n 
for ii : = 1 to i) 


for i m : - 1 to i m ~1 
k:= k+ 1 


Solution: Note that the initial value of k is 0 and that 1 is added to k each time the 
nested loop is traversed w ith a set of integers /[, i% . i m such that 

1 < i m < i m ~ \ s-aii <a 

The number of such sets of integers is the number of ways to choose m integers from 
{1,2,..., tt(, with repetition allowed. (To see this, note that once such a set has been 
selected, if we order the integers in the set in nondecreasing order, this uniquely defines 

an assignment of i m , i m -\ . i\. Conversely, ever,' such assignment corresponds to a 

unique unordered set.) Hence, from Theorem 2, it follows that k = C(n + m - 1, m) 
after this code has been executed, ■ 


The formulae for the numbers of ordered and unordered selections of r elements, 
chosen with and without repetition allowed from a set with n elements, are shown in 
Table 1. 


TABLE 1 Combinations and Permutations With and 
Without Repetition. 

Type 

Repetition Allowed? 

Formula 

r-pennutations 

No 

n[ 

(fl-7)! 

r-eombinatians 

Nu 

n\ 

r! (n - rY. 

r -permutations 

Yes 

n r 

r-combinations 

Yes 

Mr - I)! 
r\ in - 1)! 
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EXAMPLES 


THEOREM 3 


PERMUTATIONS OF SETS WITH INDISTINGUISHABLE 
OBJECTS 


Some elements, may be indistinguishable in counting problems. When this is the case, 
care must be taken to avoid counting things more than once. Consider the following 
example. 


How many different strings can be made bv reordering the letters of the word 

SUCCESS? 


Solution: Because some of the letters of SUCCESS are the same, the answer is not 
given by the number of permutations of seven letters. This word contains three A's, two 
Cs, one U, and one £. To determine the number of different strings that can be made by 
reordering the letters, first note that the three As can be placed among the seven positions 
in CO- 3) different ways, leaving four positions free. Then the two Cs can be placed in 
C(4, 2) ways, leaving two free positions. The U can be placed in C(2, 1) ways, leaving 
just one position free. Hence E can be placed in C(l, 1) way, Consequently, from the 
product rule, the number of different strings that can be made is 


C(7, ?)C(4,2)C(2,1)01. 1) 


T. 4! 2! 1! 

3i4! 2'::' 1: i: TToi 

7! 

312! 111! 

420. ■ 


Using the same sort of reasoning as in the previous example, the following theorem can 
be proved. 


The number of different permutations of n objects, where there are n\ indistinguish¬ 
able objects of type I, ni indistinguishable objects of type 2,..,, and n k indistin¬ 
guishable objects of type k, is 
b! 

nilftj! ’*■ «*! 

Proof: To determine the number of permutations, first, note that the n ] objects of type 
one can be placed among the n positions in C(n. nf ways, leaving n - n, positions free. 
Then the objects of type two can be placed in C(n - n it n 2 ) ways, leaving n - /r. L - n 2 
positions free. Continue placing the objects of type three,..., type k - 1, until at the 

last stage n k objects of type k can be placed in C(n - n\ - n 2 - n k h n k ) ways. 

Hence, by the product rule, the total number of different permutations is 

C[n, ri\)C(n - n u nf) ■■■Cln - n\ - - Uk-i-fik) 

^_(u ~ »[)! ___ (k_- «i-«*_,)! 

"i!(« - «i.)! n z !(n - «, - »,)!" £ ! O' 

__ n! _ 

rtf. nil--n k ’.' n 
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EXAMPLE 9 


THEOREM 4 


DISTRIBUTING OBJECTS INTO BOXES 


Some counting problems can be solved by enumerating the ways distinguishable objects 
can be placed into distinguishable boxes. Consider the following example in which the 
objects ape cards and the ‘'boxes" are hands of players. 


How many ways are there to distribute hands of 5 cards to each of four players from 
the standard deck of 52 cards? 


Solution: We will use the product rule to solve this problem. To begin, note that the first 
player can be dealt 5 cards in C(52, 5) ways. The second player can be dealt 5 cards 
in C(47. 5} ways, since only 47 cards are Jeft. The third player can be dealt 5 cards in 
C(42,5) ways. Finally, the fourth player can be dealt 5 cards in C(37, 5) ways. Hence, 
the total number of ways to deal four players 5 cards each is 


f 152.5)C(47,5)C(42, 5)C(37.5) 


52! 47! 42! 37! 

4775! * 42' 5' ’ 37!?! ’ 32!? 


52! 


5! 5! 5! 5! 32!' 


Remark; The solution to Example 9 equals the number of permutations of 52 objects, 
with 5 indistinguishable objects of each of four different types, and 32 objects of a 
fifth type. This equality can be seen by defining a one-to-one correspondence be¬ 
tween permutations of this type and distributions of cards to the players. To define this 
correspondence, first order the cards from 1 to 52. Then cards dealt to the first player 
correspond to the cards in the positions assigned to objects of the first type in the per¬ 
mutation. .Similarly, cards dealt to the second, third, and fourth players, respectively, 
correspond to cards in the positions assigned to objects of the second, third, and fourth 
type, respectively. The cards not dealt to any player correspond to cards in the positions 
assigned to objects of the fifth type. The reader should verify that this is a one-to-one 
correspondence, 

Example 9 is a typical problem that involves distributing distinguishable objects 
into distinguishable boxes. The distinguishable objects are the 52 cards, and the five 
distinguishable boxes arc the hands of the four players and the rest of the deck. Count¬ 
ing problems that involve distributing distinguishable objects into boxes can be solved 
using the following theorem. 


The number of ways to distribute n distinguishable objects into k distinguishable 
boxes so that n. objects are placed into box i, i ~ 1,2,..., k, equals 

n! 

»[! wj! 


The proof of Theorem 4 is left for the reader (see Exercises 43 and 44). 
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Exercises 

1. In how many different ways can five elements be se¬ 
lected in ruder from a set with three elements when 
repetition is allowed? 

2. in how many different ways can five elements be se¬ 
lected in order from a set with five elements when rep¬ 
etition is allowed? 

3. How many strings of six letters are there'? 

4. Every day a student randomly chooses a sandwich for 
lunch from a pile of wrapped sandwiches. If there are 
six kinds of sandwiches, how many different ways are 
there for the student to choose sandwiches for the seven 
days of a week if the order in which the sandwiches are 
chosen matters? 

5. How many ways are there to assign three jobs to five 
employees if each employee can be given more than 
onejob? 

6. How many ways are there to select live unordered el¬ 
ements from a set with three dements 'when repetition 
is allowed? 

7. How many ways are there to select three unordered el¬ 
ements from a set with five dements when repetition 
is allowed? 

8. How many different ways are there to choose a dozen 
donuts from the 21 varieties at a donut shop? 

9. A hage! shop has onion bagels, poppy seed bagels, egg 
bagels, salty bagels, pumpernickel bagels, sesame Seed 
bagels, raisin bagels, and plain bagels. How' many ways 
tire there to choose 

a) six bagels? 

b) a dozen bagels? 

cl two dozen bagels? 

d) a dozen bagels with at least one of each kind? 
m a dozen bagels with at least three egg bagels and no 
more than two salty bagels? 

10. A croissant shop has plain croissants, cherry croissants, 
chocolate croissants, almond croissants, apple crois¬ 
sants. and broccoli croissants. How many ways are 
there to choose 

a) a dozen croissants? 
h) Lhree dozen croissants? 

c) two dozen croissants with at least two of each kind? 

d) two dozen croissants with no more than two broc¬ 
coli croissants? 

e) two dozen croissants with at least five chocolate 
croissants and at least three almond croissants? 

f) two dozen croissants with at least one plain crois¬ 
sant, at least two cherry' croissants, at least three 
chocolate croissants, at least one almond croissant, 
at least two apple croissants, and no more than three 
broccoli croissants? 


11. How many w r ays are there to choose eight coins from 
a piggy bank containing 100 identical pennies and 80 
identical nickels? 

12. How many different combinations of pennies, nickels, 
dimes, quarters, and half dollars can a piggy bank con¬ 
tain if it has 20 coins in it? 

13. A book publisher has 3000 copies of a discrete mathe¬ 
matics book. How many ways are there to store these 
books in their three warehouses if the copies of the book 
are indistinguishable? 

14. How many solutions are there to the equation 

JL'l + Xz + X} + .14 = 17 

where xj, x 2 , x 3 , and x A are non negative integers? 

15. How many solutions are there to the equation 

JT| + *2 + Xj + Jl' 4 + Xj = 2! 

where v.. i - 1, 2,3,4,5, is a nonnegative integer such 
that 

a) x, a l? 

b) xt a 2 fori = 1.2, 3,4,5? 

c) 0 £ x, ^ 10? 

d) 0 -A xi < 3, 1 < x 2 < 4, and x 3 a 157 

16. How many solutions are there to the equation 

Jt i + Xi + + jt 4 + x -5 + ,r 6 = 29 

where x„ i = 1, 2, 3,4,5,6, is a nonnegative integer 
such that 

a) jc; > 1 fori = 1.2,3,4,5,62 

b) Xi 2 : 1, xz ^ 2, ti - 1 3, x 4 a 4, ,r s > 5, and 

Xf, a 6? 

0 *i £ 5? 

d) X] < 8 and xj > 8? 

17. How many strings of 10 ternary digits (0, 1, or 2) are 
there that contain exactly two Os, three is, and five 2s? 

18. How many strings of 20 decimal digits are there that 
contain two Os, four Is, three 2s, one 3, two 4s, three 
5s, two 7s, and three 9s? 

19. Suppose that a large family has 14 children, includ¬ 
ing two sets of identical triplets, three sets of identical 
twins, and two individual children. How many ways 
are there to seat these children in a row of chairs if the 
identical triplets or twins cannot be distinguished from 
one another? 

20. How many solutions arc there to the inequality 

X\ + X; + X} < 1 1 

where xi, x 2l and x 3 are nonnegative integers'? (Hint: 
Introduce an auxiliary variable x 4 so that t + x 2 + + 

x 4 = 11.) 
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21. How many positive integers less than 1,000.000 have 
the sum of their digits equal to 19? 

22. How many positive integere less than 1,000,000 have 
exactly one digit equal to 9 and have a sum of digits 
equal to 13? 

23. There are 10 questions on a discrete mathematics final 
exam. How many ways are there to assign scores to 
the problems if the sum of the scores is 100 and each 
question is worth at least 5 points? 

24. Show that there are C(n + r- q t - q 2 -■■■ - q r - 

- q 2 - q ,)different unordercdselections 

of n objects of / different types that include at least 
objects of type one, q 2 objects of type two,,,,, and q, 
objects of type / 

25. How many different bit strings can be transmitted if 
die string must begin with a 1 bit, must include three 
additional 1 bits (so that a total of four 1 bits is sent), 
must include a total of twelve 0 bits, and must have at 
least two 1) bits following each 1 bit? 

26. How many different strings can be made from the let¬ 
ters in MISSISSIPPI, using all the letters? 

27. How many different strings can be made from the let¬ 
ters in A BRA C.4 DA BRA , using all the letters? 

28. How many different strings can be made from the let¬ 
ters in AARDVARK. using all the letters, if all three ,4s 
must be consecutive? 

29. How many different strings can be made from the let¬ 
ters in ORONO, using some or all of the letters? 

30. How many strings with five or more characters can be 
formed from the letters in SEBRESS ? 

31. How many strings with seven or more characters can 
be formed from the letters in EVERGREEN'} 

32. How many different bit strings can be formed using six 
Is and eight 0s? 

33. A student has three mangos, two papayas, and two kiwi 
fruits. If the student cats one piece of fruit each day, and 
only the type of fruit matters, in how many different 
ways can. these fruits be consumed? 

34. A professor packs her collection of 40 issues of a math 
ematics journal in four boxes with 10 issues per box. 
How many ways can she distribute the journals if 

a) each box is numbered, so that they are distinguish¬ 
able? 

b) the boxes are identical, so that they cannot be dis¬ 
tinguished? 

35. How many ways arc there to travel in ,tv.: space from 
the origin (0, 0,0) to the point (4, 3,5) by taking steps 
one unit in She positive x direction, one unit in the pos¬ 
itive.! direction, or one unit in the positive i direction ? 
(Moving in the negative ,t, y, or j direction is prohib¬ 
ited, so that no backtracking is allowed.) 

36. How many ways are there to travel in itch 1 space from 
the origin (0,0.0,0) to the point (4,3, 5.4) by taking 
steps one unit in the positive x, positive v, positive z, or 
positive w direction? 


37. How many ways are there to deal hands of 7 cards 
to each of five players from a standard deck of 52 
cards? 

38. In bridge, the 52 cards of a standard deck are dealt to 
four players. How many different ways are there to deal 
bridge hands to four players? 

39. What is the probabil tty that each player has a hand con¬ 
taining an ace when the 52 cards of a standard deck are 
dealt to four players? 

40. In how many ways can a dozen books be placed on four 
distinguishable shelves 

a) if the books are indistinguishable copies of the same 
title? 

b) if no two books are the same, and the positions of 

the books on the shelves matter? (Him: Break this 
into 12 tasks, placing each book separately. Start 
with the sequence 1,2, 3,4 to represent the shelves. 
Represent the hooks by b h i = 1.2, ..12. Place 
b i to the right of one of the terms in 1, 2,3,4. Then 
successively place b 2 , i > 3 .and 

41. How many ways can n books be placed on k distin¬ 
guishable shelves 

a) if Lhe books are indistinguishable copies of the same 
title? 

b) if no two books are the same, and the positions of 
the books on the shelves matter? 

42. A shelf holds 12 books in a row. How many ways are 
there to choose live books so that no two adjacent books 
are chosen? (Him: Represent the btx'ks that are chosen 
by bars and the books not chosen by stars. Count the 
number of sequences of five bars and seven stars so 
that no two bars are adjacent.) 

*43. Use the product ruie to prove Theorem 4, by first plac¬ 
ing objects in the first box, then placing objects in the 
second box, and so on. 

*44. Prove Theorem 4 by first setting up a one-to-one corre¬ 
spondence betw-een permutations of n objects with rtj 
indistinguishable objects of type /, i - 1,2,3,..., k, 
and the distributions of n objects in k boxes such that m 

objects are placed in box i, i = 1,2, 3_ k and then 

applying Theorem 3. 

*45, In this exercise we will prove Theorem 2 by set¬ 
ting up a one-to-one correspondence between the set 
of /-combinations with repetition allowed of S = 

{1, 2, 3,., n} and Lhe set of /-combinations of the set 
T - {1,2,3. n + r- 1}. 

a) Arrange the elements in an r-combination, with 
repetition allowed, of 5 into an increasing sequence 
*'i — 'c s ■■■ < x r . Show that the sequence 
formed by adding k - 1 to the kLh term is strictly 
increasing. Conclude that this sequence is made up 
of r distinct elements from T. 

b) Show that the procedure described in (a) defines 
a one-to-one correspondence between the set of 
/-combinations, with repetition allowed, of .V and 
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the r- comb mat ions of T, ( Hint; Show the corre¬ 
spondence can be reversed by associating to the 
r-eorabi nation {.q. .q,,,, . x r } of 7', with 1 £ ,q < 
jt? < x, « n + r - 1, the ^-combination with 
repetition allowed from ,S, formed hv subtracting 
k - 1 tram the Aih element .I 
el Conclude that there are Cin + r - 1, r) r- 
combiiiations with repetition allowed front a set 
with u elements. 

46. How many ways are there to distribute five distinguish¬ 
able objects into three indistinguishable boxes? 

47. How many ways arc there to distribute five indistin¬ 
guishable objects into three indistinguishable boxes? 

48. How many different terms are there in the expansion 
of (q ■ v; + - - ■ - .i m )" after all terms with identical 
sets of exponents are added? 


*49. Prove the multinomial theorem: If n is a positive in¬ 
teger, then 

!>i + .V: + ■ ■ ■ + At*)* 

= X C(«;«|. «2_ 

where 


is a multinomial coefficient. 

50. Find the expansion of |.r + y + jj*. 

51. Kind the coefficient of x'vV in (a -H y + A 10 . 

52. How many terms are there in the expansion of 

f.v 1 y 4 c) l00 ‘.’ 


4 J _ _ 

Generating Permutations and Combinations 

INTRODUCTION 


Methods for counting various types of permutations and combinations were described 
in the previous sections of this chapter, but sometimes permutations or combinations 
need to be generated, not just counted. Consider the following three problems. First, 
suppose that a salesman must visit six different cities. In which order should these 
cities be visited to minimize total travel time? One way to deter mine the best order is to 
determine the travel time for each of the 6! = 720 different orders in which the cities 
can be visited and choose the one with the smallest travel time. Second, suppose some 
numbers from a set of six numbers have 100 us their sum. One way to find these numbers 
is to generate all the 2 ■ = 64 subsets and check the sum of their terms. Third, suppose a 
laboratory has 95 employees. A group of 12 of these employees with a particular set of 
25 skills is needed for a project. (Each employee can have one or more of these skills ) 
One way to find such a set of employees is to generate all sets of 12 of these employees 
and cheek whether they have the desired skills. These examples show that it is often 
necessary to generate permutations and combinations to solve problems. 


GENERATING PERMUTATIONS 

Any set with n elements can be placed in one-to-one correspondence with the set {1, 2, 2, 
.... n}. We can list the permutations of any set of n elements by generating the permu¬ 
tations of the n smallest positive integers and then replacing these integers with the cor¬ 
responding dements. Many different algorithms have been developed to generate the rt! 
permutations of this set. We will describe one oi these that is based on the lexicographic 

ordering ot the set of permutations ot [1, 2,3. nj. In this ordering, the permutation 

a'.a i -'-fla precedes the permutation of 6, /> 2 ■ ■ b n , if for some k, with i s k < n. 

«i = bi> a z - fa . Ok i - £i-i, anda* < h t . In other words, a permutation of 

the set of the u smallest positive integers precedes (in lexicographic order) a second 
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EXAMPLE 1 


EXAMPLE 2 


permutation if the number in this permutation in the first position where the two permu¬ 
tations disagree is smailer than the number in that position in the second permutation. 


The permutation 23415 of the set {1, 2,3,4,5} precedes the permutation 23514, since 
these permutations agree in the first two positions, but the number in the third position 
in the first permutation, 4, is smaller than the number in the third position in the second 
permutation, 5. Similarly, the permutation 41532 precedes 52143. M 


An algorithm for generating the permutations of {1,2,..., n} can be based on a 
procedure that constructs the next permutation in lexicographic order following a given 
permutation a : aj - - ■ flu We will show how this cart be done. First, suppose that a„. [ < 
a„. Interchange a„ [ and a„ to obtain a larger permutation. No other permutation is 
both larger than the original permutation and smaller than the permutation obtained by 
interchanging a n -1 and a n . For instance, the next largest permutation after 234156 is 
234165. On the other hand, if a„ i > a„. then a larger permutation cannot be obtained 
by interchanging these last two terms in the permutation. Look at the last three integers 
in the permutation. If a„-2 < a n -i, then the last three integers in the permutation can 
be rearranged to obtain the next largest permutation. Pul the smaller of the two integers 
a !: [ and a„ that is greater than ti „-1 in position n- 2. Then, place the remaining integer 
and a n j into the last two positions in increasing order. For instance, the next largest 
permutation after 234165 is 234516. 

On the other hand, if 2 > a,, 1 (and a n -\ then a larger permutation 

cannot he obtained by permuting the Iasi three terms in the permutation. Based on 
these observations, a genera! method can be described for producing the next largest 
permutation in increasing order following a given permutation a ^2 ■■ ■ a„. First, find 
the integers a , and a } + j with aj < a j , 1 and 

Ujt 1 > fljvj > ■■■ > a„, 

that is, the last pair of adjacent integers in the permutation where the first integer m 
the pair is smaller than the second. Then, the next largest permutation in lexicographic 
order is obtained by putting in the /th position the least integer among ■. 

and a„ that is greater than a f and listing in increasing order the rest of the integers 

(tj, £i ; * 1 . a„ in positions j + I to n. It is easy to see that there is no other permutation 

larger than the permutation a\U 2 ■ ■ a n but smaller than the new permutation produced. 
(The verification of this fact is left as an exercise for the reader.) 


What is the next largest permutation in lexicographic order after 362541? 

Solution: The last pair of integers a 3 and a i+] where a; <a J+ j is a 3 = 2 and 04 = 5. 
The least integer to the right of 2 that is greater than 2 in the permutation is ri 5 = 4. 
Hence. 4 is placed in the third position, Then the integers 2,5, and 1 are placed in order 
in the last three positions, giving 125 as the last three positions of the permutation. 
Hence, the next permutation is 364125, ■ 


To produce the «! permutations of the integers 1, 2, 3,.... n. begin with the small¬ 
est permutation in lexicographic order, namely, 123 *■ ■ n, and successively apply the 
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procedure described for producing the next largest permutation of nl - 1 times. This 
yields all the permutations of the n smallest integers in lexicographic order. 

EXAMPLE 3 Generate the permutations of the integers i. 2, 3 in lexicographic order. 

Solution: Begin with 123. The next permutation is obtained by interchanging 3 and 
2 to obtain 132. Next, since 3 > 2 and 1 < 3, permute the three integers in 132, Put 
the smaller of 3 and 2 in the first position, and then put 1 and 3 in increasing order in 
positions 2 and 3 to obtain 213. This is followed by 231, obtained by interchanging 1 
and 3, since 1 < 3. The next largest permutation has 3 in the first position, followed by 
1 and 2 in increasing order, namely, 312. Finally, interchange 1 and 2 to obtain the last 
permutation, 321, ■ 


Algorithm 1 displays the procedure for finding the next largest permutation in lexi¬ 
cographic order after a permutation that is not n n - 1 n - 2 .., 2 1 , which is the largest 
permutation. 


algorithm i Generating the Next Largest Permutation in 
Lexicographic Order. 

procedure new penmuannnUiuii ■■*«„; permutation of 
{ 1 , 2 ,..., n } not equal to n n - 1 ■ ■ ■ 2 1 ) 

>:=«-! 
while a : > a, + ] 

}■ = J- 1 

. {j is the largest subscript with a, < a ,,.[ j 
' k : = n 

j while a , > a k 

k : = k - 1 

is the smallest integer greater than a } to the right of a ,} 
interchange a 2 and a k 

I r: = " 
j j:=;f 1 

' begin 

j interchange a, and a, 

i, r:= r - 1 

j s:= v + 1 

| end 

j {this puts the tail end of the permutation after the /th position in 
I increasing order} 


GENERATING COMBINATIONS 
web —’— -— 

How can we generate all the combinations of the elements of a finite set? Since a combi¬ 
nation is just a subset, we can use the correspondence between subsets of {a\, a„) 

and bit strings of length n. 
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EXAMPLE 4 


EXAMPLE 5 


Recall that the bit string corresponding to a subset has a 1 in position k if a* is in 
the subset, and has a 0 in this position if a* is not in the subset. If all the bit strings of 
length n can be listed, then by the correspondence between subsets and bit strings, a 
list of all the subsets is obtained. 

Recall that a bit string of length «is also the binary expansion of an integer between 
0 and 2" - 1. The 2" bit strings can be listed in order of their increasing size as integers 
in their binary expansions. To produce all binary expansions of length n, start with the 
bit string 000.. .00, with n zeros. Then, successively find the next largest expansion 
until the bit string 111... 1 i is obtained. At each stage the next largest binary' expansion 
is found by locating the first position from the nght that is not a 1, then changing all the 
is to the right of this position to 0s and making this first 0 (from the right) a 1. 


Find the next largest bit string after 10 OOIG 0111, 

Solution: The first bit from the nght that is not a 1 is the fourth bit from the right. 
Change this bit to a 1 and change all the following bits to Os. This produces the next 
largest bit string, JO 0010 1000. g 


The procedure for producing the next largest bit string after b H - t b„_ 2 . b[h n is given 
as Algorithm 2. 


L_ 


AI.tiORlTHM 1 Generating the Next Largest Bit String. 

procedure next bit stringib,, { b„-i ,.. bit string net equal to 11 ... 11) 

i: = 0 

whiled, - 1 
begin 

b,:= 0 

1 : = f f 1 

end 

bi:-\ 


Next, an algorithm for generating the r-combinations of the set {1, 2, _1__«} wii! 

be given. An /-combination can be represented by a sequence containing the elements 
in the subset in increasing order. The /-combinations can be listed using lexicographic 
order on these sequences. The next combinations alter a\t /; can be obtained in the 

following way: First, locate the last element a, in the sequence such that -.t, x n / • i 

Then, replace u, with a, + 1 and a s with a, + j - i +F for j = i+l P i' + 2. r. It is left 

for the reader to show that this produces the next largest combination in lexicographic 
order. This procedure is illustrated with the following example. 


Find the next largest 4-combination of the set {1. 2. 3 4, 5, 6} after f 1, 2, 5, 6}. 

Solution: The last term among the terms a, with u, f. a, - 2.u 3 - A and u± - 6 
such that u,- ^ 6 - 4 + / is a 2 - 2, To obtain the next largest 4-combi nation, increment 
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a -2 by 1 to obtain aj - 3. Then set a\ - 3 + 1 = 4 and -3 + 2 = 5. Hence the 
next largest 4-eombination is {1,3,4, 5}. ■ 


Algorithm 3 gives this procedure in pseudocode. 


algorithm J Generating the Next /-Combination in Lexicographic 
Order. 

procedure next r-combinatum({a\M 2 ,. •,«,}; proper subset of 
{1.2,_«} not equal to {n - r + 1.«} with 

A; < a2 < ■• ■ < m) 
i: - r 

while a, - n - r + i 
(':=/■- I 
n r : - ai + 1 
for / : - i + I to r 

Or : — a, + j - i 


Exercises 


1. Find die next largest permutation in lexicographic or¬ 
der after each of the following permutations. 

a) 1432 hi 54123 

c) 12453 4J 45231 

e) 0714235 f) 31528704 

2. Place the following pelimitations of {1.2,3.4,5,6} in lex¬ 
icographic order: 234561, 231456, 165432, 156423, 
543216, 541236, 231465, 314562, 432561, 654321. 
654312, 435612. 

3. Use Algorithm I lo generate the 24 permutations of the 
First four positive integers in lexicographic order, 

4. Use Algorithm 2 to iisi all the subsets of the set 
11.2, .3.4}. 

5. Use Algorithm ,3 to list al! the 3-combinations of 
{1,2,3, 4.5}. 

6. Show that Algorithm 1 produces the next largest per¬ 
mutation in lexicographic order 

7. Show that Algorithm 3 produces the next largest 
/-combination in lexicographic order after a given 
/■-combination. 

ft. Develop an algorithm for generating the r- 
pcmiutarions of a set of /; elements. 

9. List all 3-permutations of {1, 2,3,4,5}. 

The remaining exercises in this section develop another al¬ 
gorithm for generating the permutations of {1,2, 3,.,., n}. 
This algorithm is based on Cantor expansions of integers. 


Every nonnegative integer less than tt! has a unique Canter 
expansion 

«i U T ai2* + ••■ + a n _i(« -- 1)1 
where ct, is a nonnegative integer not exceeding i, for t ~ 

1.2 . n- 1. The integers Q[, a 2 . a „. L are called the 

Cantor digits of hits integer. 

Given a permutation of {1,2,..., m, let a t _|, it =• 

2.3 . n, be the number of integers less than k that fol¬ 

low k in the permutation. For instance, in the permutation 
43215, m is the number of integers [ess than 2 that fol¬ 
low 2, so that ay = 1. Similarly, for this example a? - 2, 
ai = 3, and m = 0, Consider the function from the set of 
permutations {1,2,3,.... n} to the set of nonnegative inte¬ 
gers less than n ! that, sends a permutation to the integer that 
has a [, « 3 .tin-i. defined in this way, as its Cantor digits. 

10, Find the integers that correspond to the following per¬ 
mutations. 

a) 246531 b) 12345 cl 654321 
*11, Show that the correspondence described here is a bflec¬ 
tion between the set of permutations nf {!, 2,3, 
and the nonnegative integers less than n!. 

12, Find die permutations of {1, 2, 3,4.5} that correspond 
to the following integers with respect to the correspon¬ 
dence between Cantor expansions and permutations as 
described before Exercise 10, 

□) 3 b) 89 c) 111 
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13. Develop an algorithm for producing all permutations 
of a set of n elements based on the correspondence dc 
scribed in the preamble to Exercise 10, 

* [4. The fol lowing method can be used to generate a random 
permutation of a sequence of n terms. First, interchange 
the nth term and the rffrith term where riii) is a ran¬ 
domly selected integer with 1 s tin) £ Next, in¬ 
terchange the (a - 1 )th term of the resulting sequence 
with its r(« - 1 }th term where r(n - J) is a randomly se - 
leeted integer with 1 r(n-l) s «- i. Continue this 


process until j -- n , where at the j th step you inter¬ 
change the i r - j t I )lh term of the resulting sequence 
with its r{n - j + } )tb term, where r\ n - j + 1 ) is a 
randomly selected integer with 1 £ rftt - / + 1 ) s 
n~ j + i. Show 1 that when this method is followed 
each of the n! different permutations of the terms of the 
sequence is equally likely to be generated, [Hint: Use 
mathematical induction, assuming that the probability 
that each of the permutations of n - 1 terms produced by 
this procedure for a sequence of n- (terms is \.Hn l)!j. 


Key Terms and Concepts 


I EKT1S 

coirthinalories: the study of arrangements of objects 
enumeration: the counting of arrangements of objects 
tree diagram : a diagrut n made up of a root, branches leav¬ 
ing the root, and other branches leaving some of the end- 
points of branches 

permutation: an ordered arrangement of the dements of 
a set 

r-pennufatiini: an ordered arrangement of r elements of a 
set 

Fin. r): the number of r-pci mutations of a set with tt de¬ 
ments 

/-combination: atronordered selection ofrdeineiitsotaset 
C{n, f}: the number of r-eombinations of a set with n ele- 
meins 

{" (binomial coefficient): also the number of r-eom 
' ' binations of a set with /i elements 
Pascal’s triangle: a lepresentatioti of the binomial etteffi- 
cients where the ;'th row of the mangle contains CU. j ) 
for j - 0, 1,2,_ i 

probability of an event: the number of successful out¬ 
comes of this event divided by the number of possible 
outcomes 

p(E \ F) (conditional probability of F given F): ptE : 

i 'hV ) 

independent events: events £ and (’.such that p{F. C\F\ 
p{E)p\F) 

random variable: a function that assigns a real number to 
each outcome of an experiment 
expected value of a random variable: the weighted av¬ 
erage of a random variable, with values of the random 
variable w sighted by the probability of outcomes, that is, 
F{X] - /•(*.}•*(>'.) 

variance of a random v ariable: the weighted average of 
die square of the difference between the value of the 
random variable and its expected value, with weights 
given by the probability of outcomes, that is, V(A') ■- 


Bernoulli trial: an experiment with two possible out¬ 
comes 

RESULTS 

The sum rule: a basic counting technique which states that 
the number of ways to do a task in one of two ways is the 
sum of the number of ways to do these tasks if (hey cannot 
be done simultaneously 

The product rule: a basic equating technique which states 
that the* number of ways to do a procedure that consists 
of two subtasks is the product of the number of ways to 
do the first task and the numher of ways to do the second 
task after the first task has been done 

The pigeonhole principle: When more than k objects -j its 
placed in k boxes, there must be a box containing more 
than one object. 

The generalized pigeonhole principle; When N objects 
are placed in k boxes, there must be a box containing at 
least [jV/A' 1 objects. 


Cin r) - n ' {n - rtf 

r\ 

Pascal’s identity: C{n l,i) m C(n, k - l j + Cin. k) 
The binomial theorem: (jt + y)“ = S!** 0 C(n. t)at* 

The probability of k successes when n independent 
Bernoulli trials are carried out equals C(n. k)p l q"~ l , 
where p is the pvobabiliiy of success and q — 1 - pis 
the probability of failure, 

There are n r r-penmitatioiis of a set with n elements when 

repetition is allowed. 

There are C(n + r \ j) r-imribifiatiftni of a set with >i 
dements when repetition is allowed 
Then': are n!/(rq! n 2 :'--,*P!) permutations of n objects 
where there are indistinguishable objects of type • for 

f- 1,2,3.. k: 

The algorithm for generating the permutations of the set 

{1.2,---.4 
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Review Questions 


1. Explain hum- the sum and product rules can be used to 
find the number of bit strings with a length not exceed¬ 
ing 10. 

2. Explain how to find the number of bit strings of length 
not exceeding 10 ihat have at least one 0 bit, 

3. at How can the product rule be used to find the number 

of functions from a set with in elements to a set with 
‘t elements'? 

b> How many functions axe there from a set with 5 el¬ 
ements to « set with 10 eiemenis? 
v 1 l-fcwcan the product rule be used to find the number 
a!'one-to-one functions from a set with m elements 
■ n $ sei fi elements'’ 

ri) How mail) one-to-one functions are there from a set 
with 5 elements to a set with it) elements'? 
e) How many onto functions are there from a set with 
5 elements to a set with 10 elements? 

4. \ low cun vt.m iirsd the number of possible outcomes of 
a piaveif between (wo teams where the first team that 
win 1 ' tour games wins the playoff 1 

5. How cap you bud the number of Ml strings of length 
1 r 11bat either begin with 1(1! or end with 010? 

b, .i fc Suite the pigeonhole principle, 

i>S bow the pigeonhole principle can be used 

ro ‘-him that among any ! 1 integers, at least 2 must 
r*. • die same last digit. 

7. a i Mat.; tin* generalized pigeonhole principle. 

hi lixpi,!iti how me generalized pigeonhole principle 
v.r; K- used to show that among any 91 integers, 
ro.-rv are ;.t least 10 that end wiUi the same digit. 

v - 1 tvt.ji r, pic difference be'w,-i*i\ an r-combination 
.tii' f :m j ■ pen nutation if c wit with n elements ? 
i.-i iretm 1 .m equati.-n that relates the number of 
r t'emhinidionn and the number of r-permuiations 
>-f .i •*! with a dement:-,. 

. 1 1 'us. many way:»are there to ■,-elect 6 students from 
Sclavs of 25 to serve oil a committee'? 

(it How pidtiy ways arc there to select fi students from 
o class of 25 in hold six different rxccuti ve positions 
■>£t .1 cu-w.iiiee? 

b. mi Wfur F Hiiscarb txiangle' 1 

lil How can a row- tit' Pascal's triangle be produced 
»'r«rr. me one above it? 

1(1. Mu; is mean! by a combinatorial proof {if an identity? 
How i:> such a proof different from an algebraic one'.’ 

11. Explain how to prove: Pascal's identity using a comtn- 
antonal argument. 

S3 at V.r.-. 'ini binomial theorem. 

b? l-.spuia Ijuw to prove the binomial theorem using a 
--v.m«nawnal argument. 


cl Find the coefficient uf .r !a V 01 in the expansion of 
(2x -+ .Vi 201 . 

13. a) Deline die probability of an event when all out¬ 

comes are equally likely. 

b) What is the probability that you select the six win¬ 
ning numbers in a lottery if the six different win¬ 
ning numbers are selected from the first 50positive 
integers? 

14. a) What conditions should be met by the probabili¬ 

ties assigned to the outcomes- from a finite sample 
space' 1 

bj What probabilities shun Id be assigned to the out¬ 
come of heads and the outcome of tails if heads 
crimes up three times as often as tails? 

15. a) Define the conditional probability of an event E 

given an event F. 

b) Suppose £ is the event that when a die is rolled it 
comes up an even number, and F is the event that 
when a die is rolled it comes up 1,2, or 3. What is 
the probability of £ given £'? 

16. a) When arc two events E and /'independent? 

bl Suppose l is the event that an even number appears 
when j fair die is rolled, and F is the event ihat a 5 
or 6 unties up. Are E and F independent? 

17. a) W'hat is a random variable?' 

hi What arc the values assigned by the random vari¬ 
able X that assigns, to ,v toll of two dice the larger 
number that appears on the two dice? 

18. at Define the expected v alue of a random variable A", 
bt VVfi.it is the expected value of the random variable 

A that assigns to a roll of two dice the larger rtumbei 
that appears on the two dice? 

19. a) f; xpiain how the a ve race-case computational com¬ 

plex?'' of an algorithm, with finitely many possi¬ 
ble input values, can be interpreted as a/i expected 
value. 

h) What i s the average-i:ase computational complex ity 
(if the linear search algorithm, if the probability that 
the element for which we search is m the list is 1 t.i, 
and tt is equally likely that this element i- any rl 
the tt elements in the list? 

20. a) What is meant, by a Bernoulli trial? 

bi Wha! is the probability of k successes in n indepen¬ 
dent Bernoulli trials? 

ct What is the expected value of the number of sue 
wv- in n independent Bernoulli trials? 

21. a) What is the variance of a random variable ’ 

b) What is the variance ct a Bernoulli trial with prch 
ability /> of success? 
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22, a) What is the variance ot tbe sum of flitldepedent ran¬ 

dom variables? 

b) What is the variance of the number of successes 
when n independent Bernoulli trials, each with 
probability p of success, axe carried out? 

23, a) Explain how to find a formula for the number of 

ways Eo seleci r objects from n objects when rep¬ 
etition is allowed and order does not matter, 

III How many ways arc there to select a dozen objects 
luwn among objects of live different types if objects 
of the same type are indistinguishable? 
e) How many ways are there to select a dozen objects 
from these five different types if there must be at 
least three objects of the first type? 
tit How many w ays are there to select a dozen objects 
from these five different types if there cannot be 
more than tour objects of the firs! type? 
ft How many ways arc there to select a dozen objects 
front these five different types if there must be at 
least two objects of the first type but no more than 
three objects of the second type? 

24, a) Let n and r be positive integers. Explain why the 

number of solutions of the equation Xj ■+ Xj 4- " ■ + 


x r = r, where ay is a nonnegative integer for i = 
1,2, 3, — n, equals the number of r-combinaiions 
of a set with a elements. 

h) How many solutions in nonnegative integers are 
there to the equation ,t| + .ly + x 7 + ,r. ( - 17? 

c) How many solutions in positive integers are there to 
the equation in part (b) 7 

25. a) Derive a formula for the number of permutations 

of n objects of k different types where there are «] 
indistinguishable objects of type one, n 2 indistin¬ 
guishable objects of type two.and n* indistin¬ 

guishable objects of type k. 

b) How many ways are there to order the letters of the 
word INDISCREETNESS1 

26. Describe an algorithm for generating all the permuta¬ 
tions of the set of the n smallest positive integers, 

27. ai How many ways are there to deal hands of 5 cards 

to six players from a standard 52-card deck? 
b) How many ways are there to distribute n distm 
guishablc objects into k distinguishable boxes so 
that n, objects are placed in box i? 

28. Describe ati algorithm for generating all the combina¬ 
tions of the set of the n smallest positive integers, 


Supplementary Exercises 


1. How many ways are there to choose 6 items from 10 
distinct items when 

a) the items in the choices are ordered and repetition 
is not allowed? 

b'i the items in the choices are ordered and repetition 
is allowed? 

c) the Hems in the choices are unordered and repetition 
is not allowed? 

d) the items in the choices are unordered and repetition 
is allowed? 

2. How many ways are there to choose 10 items from b 
distinct items when 

a) the items in the choices art* ordered and repetition 
is not allowed? 

b) the items in the choices are ordered and repetition 
is allowed? 

c i the items in the choices are unordered and repetition 
is not altoured? 

d) the items in the choices are imordered and repetition 
is allowed? 

3. A test contains JCHitouednlse questions.How- many dif¬ 
ferent ways can a student answer the questions on the 
test, if answers may be lef t blank? 

4. How- many bit strings of length 10 either start with 000 
or end with 1111 ? 

5. How many bit strings of length I Dover the alphabet 


[a. b. ef have either exactly three us or exactly four 6s? 

6, The iniemal telephone numbers in the phone system 
on a campus consist of five digits, with the first digit 
not equal to zero. How many different numbers can be 
assigned in this system? 

7, An icc cream parlor has 28 different flavors, 8 different 
kinds of sauce, and 12 toppings. 

a) In how many different ways can a dish of three 
stoops of ice cream be made where each flavor can 
be used more than once and the order of die scoops 
does not matter? 

b) How many different kinds of small sundaes are 
there if a small sundae contains one scoop of ice 
cream, a sauce, and a topping? 

cl How many different kinds of large sundaes are there 
if a large sundae contains three scoops o! ice cream, 
where each flavor can be used more than once and 
die order of the scoops does not matter; two kinds of 
sauce, where each sauce can be used only l nee and 
the order of Lite sauces does not matter, and mice 
toppings, when; each topping can be used only once 
and the order of the toppings does not matter? 

8, How many positive integers less than 1000 

a) have exactly three decimal digits? 

b) have an odd number of decimal digits? 

cl have ai least one decimal digit equal to D? 
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d) have no odd decimal digits? 
el have two consecutive decimal digits equal to 5? 
fl arc palindromes (that is, read the same forward and 
backward I? 

9, When the numbers front l to 1000 are written out in 
decimal notation, how many of the following digits are 

li WStP 

si 0 hi i ct 2 d) 9 

lft. There are S3 sisns of She zodiac. How many people 
arc needed to guarantee dial a! least sis of these people 
have the same sign? 

11. A fortune cookie company makes 213 different for¬ 
tunes. A student cats at a restaurant that uses fortunes 
front this company. What is the largest possible num¬ 
ber of times that the student can eat at the restaurant 
without getting the same fortune four times? 

12. How man) people arc needed to guarantee that at least 
two were bom on the same day of the week and in the 
same month (perhaps in different years)? 

1,1. Show tlud there are at least two different five-element 
sah'eis of a set of 10 positive integers not exceeding 50 
that have the same sum. 

14. package pf baseball cards contains 20 cards. How 
many poor:age.:; must he purchased to ensure that two 
c uds m these packages am identical if there arc a total 
cf 550 different cards? 

15. a) How many cards must be chosen from a deck to 

guar fin tec that at Icaat two arcs arc chosen? 
h> How many can's must be chosen from a deck to 
guarantee tit.;! at ’east two aces and two kinds arc 
c hotcu ? 

c) How many cards must be chosen from a deck to 
guarantee that there are at least two cards of rite 
same kind? 

(1) How many cards must be chosen from a deck to 
guarantee that there are at least two cards of two 
different kinds’.' 

*16. Show that in any set of rt + 1 positive integers not ex¬ 
ceeding 2n there must he two that arc relatively prime. 

1 17. Show that in it sentience of m integers there exists one 
or more consecutive terms with a. sum divisible by in. 

IS. Show that if live points are picked in the interior of a 
square w ith a side length of 2. then at least two of these 
points arc no farther than J 2 apart. 

19. Show that the decimal expansion of a rational number 
must repeat itself from some point onward. 

20. How many diagonals does a regular polygon with n 
sides have, where n is a positive integer with n s 3'? 

21. How many ways are there to choose a dozen donuts 
front 20 varieties 

a) if there arc no two donuts of the same variety? 

b) if all donuts are of the same variety ? 

c) if there arc no restrictions? 

d) if there are at least two varieties? 

e) if there must be at least six blueberry-filled donuts? 


0 if there can be no more than six blueberry-filled 
donuts? 

22. What is the probability that six consecutive numbers 
■will be chosen as the winning numbers in a lottery 
where each number chosen is between i and 40 (inclu¬ 
sive)? 

23. What is the probability that a hand of 13 cards contains 
no pairs? 

24. Find n if 

a) Pin, 2) - HQ. b) Pyn.n) =■ 5040, 
c) Pin, 41 - 12Flu. 2). 

25. Find n if 

a) C(n, 2) = 45. b) C(n, 3) = Pin, 2). 
c) C{n, 5) = Cfe 2). 

26. Show that if n and r are nonnegative integers and n =r 
r, then 

Pin + 1, r) - Pin, r)(n + l>/(n + 1 - r). 

27. Give a combinatorial proof that Cin, r) -- Cin. n - r). 

28. Give a combinatorial proof of Theorem 7 of Section 4.3 
by setting up a correspondence between the subsets of 
a set with an even number of elements and the subsets 
of this set with an odd number of elements, ( Hint: Take 
an element a in the set. Set up the correspondence by 
putting (i in the subset if it is not already in it and taking 
it out if it is in the subset.) 

29. Let n and r be nonnegative integers with r < n. Show 
l hat 

C(n,i■ - it - C.(n + %r + I) 

- 2C(n + 1, r +■ 1) t C{n. r + lr 

30. Prove using mathematical induction that S ' 2 Ci i, 2) 
=■ Cin + 1,3) whenever n is art integer greater 
than 1 , 

31. Use the binomial theorem to prove that 3" = 

k)2 l . (Hint: Let x = 1 and y ~ 2 in the 
statement of the theorem.) 

32. )n this exercise we will derive a formula for the sum of 
the squares of the n smallest positive integers. We will 
count the number of triples (i, j, ki such that/, and 
k tire integers such that 0 ^ i < t, 0 £ j < k, and 
1 S k rc ri in two ways. 

a) Show thar there are k 1 such triples with a fixed k. 

Conclude that there are 23such triples, 
h) Show that the number of such triples with 0 £ i < 
j < k and the number of such triples with 0 < j < 
i < k both equal C(n + 1 ,3). 

c) Show that the number of such triples with 0 s i 
j < k equals C{n + 1,2). 

d) Combining pan fa) with parts fb) and fcl conclude 
that 

= 2C(n+ J, 3) + C(n +1.2) 

k--- 3 

= n(ft+ l)(2n + I)/6. 
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*33, H» w many bit strings of length n ,.where n 2 4, contain, 
exactly two occurrences of 01? 

34, What is the probability that a seven-card poker hand 
contains 

8) four cards of one kind and three cards of a second 
kind? 

h i three cards of one kind and pairs of each of two dif¬ 
ferent kinds? 

c) pairs of each of three different kinds and a single 
card of a fourth kind? 

d) pairs of each of two different kinds and three, cards 
of a third, fourth, and fifth kind? 

e) cards of seven different kinds? 

f) a seven-card flush? 

K) a seven-card straight? 

hi a seven-card straight flush? 

35, What is the probability that a 13-card bridge hand con¬ 
tains 

a) all 13 hearts? 

b) 13: cards of the same suit? 

c) 7 spades and 6 clubs? 

d) 7 cards of one suit and 6 cards of a second suit? 
f) 4 diamonds, 6 hearts, 2 spades, and 1 club? 

fl 4 cards of one suit, 6 cards of a second suit, 2 cards 
of a third suit, and 1 card of the fourth suit? 

36, Suppose that p and q are primes and n = pq What is 
the probability that a randomly chosen positive integer 
less than n is not divisible by either p or q‘l 

*37, Suppose that m and n are positive integers. What is the 
probability that a randomly chosen positive integer less 
than run is not divisible hy either nt or nl 

38, Suppose that E\, E: . £„ are n events with p{E f ) > 0 

for 1 -- 1,2,.,., n. Show that 

p(E, n t'i n ■ - - n fj 

= p{£i)p(Er | £, )p(E 3 J £, n Ei) 
■■■/'(£„! Ei n£, n ■ ■ n£„ ,). 

39. We say that the events Ei. Ej ,..., E„ are mitfu 
ally independent if ptf,, n £, n ■■■ n £,„) = 

/>(£,, )p(E l .,)--- p(£,J whenever i jt j = 1,2_ m, 

are integers with i £ i ( < fj < < t m < n and 

m > 2." 

ai Write out the conditions required for three events 
E|. £:. Ei to be mutually independent. 

It I for Ei. E}, and £1 be the events that the first flip 
comes up heads, that the second flip comes up tails, 
and that the third (lip comes up tails, respectively, 
when a fair coin is flipped three limes. Are £,, £>, 
and Ei mutually independent? 
cl Let Ei. E z . and £, be the events that the first flip 
comes up heads, that the third flip comes up heads, 
and that an even number of heads come up, respec¬ 
tively, when a fair coin is flipped three times. Are 
Ei , Ez. and £j mutually independent 1 ’ 


d> How many conditions must be checked to show that 
n events are mutually independent? 

*41), Suppose that E and F are events with p(F) s* 0, Show 
(hat the probability of £ is the weighted average of the 
probability of £ given F and the probability of £ given 
the complement of F, F. where the weights are the 
probabilities of F and F, respectively. That is. 

p(E) = f>(£ i F)p(F) + p(E ] FiplFl 

l Him: Use the fact that £ = (E n F) u (£ n F).) 

*41, Suppose that E is an event from a sample space 5 and 
that F i, Ej, are mutually exclusive events such 
that U jLx F t - S. Assume that piE) E band p.\F,\ # 
0 for i = 1,2,..,, ti. Show that 


Pifj f F) 


p{E j Fj )p(F,) 

: , P(£ I F.'PiF I 


[Hint: Use the fact that E = U " = , (£ n £)■] This result 
is known as Bayes’ formula, since it was developed by 
the English philosopher 'Thomas Bayes, 

*42, A space probe near Neptune communicates with Earth 
using bit strings. Suppose that in its transmissions it 
sends a I one- third of the time and a 0 two-thirds of 
the time. When aO is sent, the probability it is received 
correctly is 0.9, and the probability it is received incor¬ 
rectly (as a I j is 0,1. When a 1 is sent, the probability 
it is received correctly is 0,8, and the probability it is 
received incorrectly (as a {)) is 0.2. 

3 } Use Exercise 40 to find the probability that a 0 is 
received. 

b) Use Bayes' formula, given in Exercise 41, to find 
the probability that a 0 was transmitted, given that 
a 0 was received. 

43. Let X be a random variable on a sample space 5. Show 
that V(aX + b) = or V{X) whenever a and b are real 
numbers. 

44. Show that if m is a positive integer, then the probability 
that the mth success occurs on the [m + pith trial when 
independent Bernoulli trials, each with probability p of 
success, are run, is j " 4 * ! ” * jp"p"\ 

45. A professor writes 20 multiple-choice questions, each 
with the possible answer a , b, c, or d. for a discrete 
mathematics test. If the number of questions with a, b. 
c. and d as their answer is 8, 3, 4, and 5, respective¬ 
ly, how many different answer keys are possible, if the 
questions can be placed in any order? 

46. How many different arrangements are there of eight 
people seated at a round table, where two arrangements 
are considered the same if one can be obtained from the 
other by a rotation? 

47. How many ways arc there to assign 24 students to live 
faculty advisors? 

48. How many ways are there to chew sc a dozen ap¬ 
ples from a bushel containing 20 indistinguishable 
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Delicious apples, 20 indistinguishable Macintosh ap¬ 
ples, and 20 indistinguishable Granny Smith apples, if 
at least three of each kind must be chosen? 

49. How many solutions are there to the equation sty +_r- + 
-i : ■■= 17, where ,iy, X 2 , and Jtj are oonnegative integers 
with 

a) jci > I, vs > 2, and x% > 3? 

b) t[ < 6 and .13 > 5? 

c) JT] < 4, .ty < 3, and ay > 5? 

50. a) How many different strings can be made from the 

word PEPPERCORN when ail the letters are used? 

b) How many of these strings start and end with the 
letter P' ! 

c) lit how many of these strings are the three letter Pb 
consecutive? 

51. How many subsets of a set with 10 elements 

a) have fewer than 5 dements ' 1 


b) have more than 7 dements? 

c) have an odd number of elements? 

52, A witness to a hit-and-run accident teils the police 
that the license plate of the car in the accident, which 
contains three letters followed by three digits, starts 
with the letters AS and contains both the digits 1 and 2. 
How many different license plates can lit this descrip¬ 
tion'? 

53, How many wavs are there to put n identical objects into 
m distinct containers so that no container is empty ? 

54, How many ways are there to seat six buys and eight 
girls in a row of chairs so that no two boys are seated 
next to each oiiier? 

55, Devise an algorithm for generating all the /-per¬ 
mutations of a finite set w hen repetition is allowed, 

56, Devise an algorithm for generating aJ! the r-com- 
binaiions of a finite set when repetition is allowed. 


Computer Projects 

WRITE PROGRAMS WITH THE FOLLOWING INPUT AND OUTPUT. 


1. Given a positive integer n and a nonnegative integer 
not exceeding n, find the number of r-permulations and 
/combinations of a set with a elements. 

2. Given positive integers n and r, find the number 
oi /-permutations when repetition is allowed and 
/■-combination'! when repetition is allowed of a set 
with n dements. 

3. Given a positive integer n, find the probability of se¬ 
lecting the six integers from the set { 1 , % , ,n} that 
were mechanically selected in a lottery. 

4. Given a sequence of positive integers, find the longest 
increasing and the longest decreasing subsequence of 
the sequence. 

5. Simulate repeated trials of the Monty Hall Three Door 
problem to calculate the probability of winning w ith 
each .strategy, 

* 6 . Given an equation ty +■ tj + ■■• + *„ = C, where (7 is 
a constant, and vy, t-... . are oonnegative integers, 
list all the volutions. 


7. Given a positive integer «, list ail the permutations of 
Ibe set { 1 , 2 . 3...., n) in lexicographic order. 

8 . Given a positive integer n and a nonnegative integer r 

not exceeding n, list all the /-combinations of the set 
|l.2,3. 11 } in lexicographic order. 

9. Given a positive integer n and a nonnegative integer / 
not exceeding n, list all the /-permutations of the set 
{ 1 , 2 ,3,,.., n} in lexicographic order. 

10, Given a positive integer n, list all the combinations of 

the set{l, 2 ,3.«}. 

11. Given positive integers n and /. list all the /-per¬ 
mutations, with repetition allowed, of the set 
{1,2,3,.. ,n}. 

12. Given positive integers n and r, list all the /-com¬ 
binations. with repetition allowed, of the set 
{1.2.3 

13, Given a positive integer n, generate a random pennuta- 
tionofthe .se! {1,2,3,.., «}. (See Exercise 14 in Near-it 
4.7.) 


Computations and Explorations 

USE A ('(JMI'LTATIONAL PROGRAM OR PROGRAMS Nv/L.' 
i ML toi l )A Rt |S[ V 

h Find the number of possible outcomes in a two-team 
phiyuf! when the winner is the first learn to win 5 out 
to % n out of 11, 7 nut of 13, and 8 out of 15 


IAVH WRITTEN TOGO 

2. Which binonna! coefficients are odd? Can you formu¬ 
late a conjecture based on numerical evidence ? 

3. it is not known whether the binomial coefficient 
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(\.2r. ni must be divisible by the square of a prime 
or whether the largest exponent in the prime factoriza¬ 
tion of C{2n, n) grows without bound as n grows. Ex¬ 
plore these questions by finding the smallest and largest 
powers of primes in the factorization of C(2n, n) for as 
many positive integers n as feasible. 

4. Find the probabilities of each type of hand in five-card 
poker and rank the types of hands by their probability. 

5. Find some conditions so that the expected value of buy¬ 
ing a SI lottery ticket in the New Jersey pick-six lot¬ 
tery has an expected value of more than $ 1 . To win you 
have to select the six numbers draw n, where order does 
not matter, from the positive integers 1 to 48, inclusive. 
The winnings are split evenly among holders of win¬ 
ning tickets. Be sure to consider the total size of the 
pot going into the drawing and the number of people 
buying tickets. 

6. Estimate the probability that two integers selected at 
random are relatively prime by testing a large number 


of randomly selected pairs of integers. Look up the the¬ 
orem that gives this probability and compare your re¬ 
sults with the correct probability, 

7. Determine the number of people needed to ensure that 
the probability at least two of them have the same day 
of the year as their birthday is at least 70 percent, at 
least 80 percent, at least 90 percent, at least 95 percent, 
at least 98 percent, and at least 99 percent. 

8. Generate all the permutations of a set with eight 
elements 

9. Generate all the <>•-permutations ot a set with nine 
elements, 

10 . Generate all combinations of a set with eight elements. 

11 . Generate all 5-combinations with repetition allowed of 
a set with seven elements. 

12. Generate a list of 100 randomly selected permutations 
of the set of the first 100 positive integers. (See Exer¬ 
cise 14 to Section 4.7.) 


Writing Projects 

RESPOND: it) THE FOLLOWING WITH liSSAVS USING OUTSIDE SOURCES. 


1. Describe some of the earliest uses of the pigeonhole 
principle by Dhichlet and other mathematicians. 

2. Discuss ways in which the current telephone number¬ 
ing plan can be extended to accommodate the rapid de¬ 
mand for more telephone numbers. (Sec if you can find 
some of the proposals coming from the telecommuni¬ 
cations industry.) For each new numbering plan you 
discuss, show how to find the number of different tele¬ 
phone numbers it supports 

3. Many combinatorial identities are described in this 
book Find some sources of such identities and de¬ 
scribe important combinatorial identities besides those 
already introduced in this book. Give some represen¬ 
tative proofs, including combinatorial ones, of some of 
these identities. 

4. Describe the origins of probability theory and the first 
uses of this theory 

5. Describe the different bets you can make when you 
play roulette Find the probability of each of these bets 
in the American version where the wheel contains the 
numbers 0 and 00. Which is the best bet and which is 
the worst for yon 11 

6. Discuss the probability of winning when you play the 


game of blackjack versus a casino. Is there a winning 
strategy for the person playing against the house? 

7. Describe the different models used to model the dis¬ 
tribution of particles in statistical mechanics, includ¬ 
ing Maxwell-Bol tom aim, fkise-Finstein, and Fermi- 
Dirac statistics. In each case, describe the counting 
techniques used in the model 

8. Define the Stirling numbers of the first kind and de¬ 
scribe some of their properties and the identities they 
satisfy. 

9. Define the Stirling numbers of the second kind and de¬ 
scribe some of their properties and the identities they 
satisfy, 

10 . Define Ramsey numbers, state and prove the theorem 
of Ramsey that shows they exist, and describe what is 
currently known about them. 

11. Describe additional ways to generate all the permuta¬ 
tions of a set with n elements besides those found in 
Section 4.7, Compare these algorithms and the the al¬ 
gorithms described in the text and exercises of Section 
4.7 in terms of their computational complexity. 

12. Describe at least one way to generate all the partitions 
of a positi ve integer n. (See Exercise 35 in Section 3.3.) 
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M any counting problems cannot be solved easily using the methods discussed 
in Chapter 4. One such problem is: How many bit strings of length n do not 
contain two consecutive zeros? To solve this problem, let a„ be the number 
of such strings of length n. An argument can be given that shows a„*i = a n + u„-[. 
Tills equation, called a recurrence relation, and the initial conditions a\ =2 and aj = 3 
determine the sequence Moreover, an explicit formula can be found for a,, from 
the equation relating the terms of the sequence. As we will see, a similar technique can 
be used to solve many different types of counting problems. 

We will also see that many counting problems can be solved using formal power 
series, called generating functions, where the coefficients of powers of .c represent terms 
of sequence wc are interested in. Besides solving counting problems, we will also be 
able to use generating functions to solve recurrence relations and to prove combinatorial 
identities. 

Many other kinds of counting problems cannot be solved using the techniques dis¬ 
cussed in Chapter 4, such as: How many way s are there to assign seven jobs to three 
employees so that each employee is assigned at least one job? How many primes are 
there less than 1000? Both of these problems can be solved by counting the number 
of elements in the union of sets, We will develop a technique, called the principle of 
inclusion-exclusion, that counts the number of elements in unions of sets, and we will 
show how this principle can be used to solve counting problems. 

The techniques studied in this chapter, together with the basic techniques of Chap¬ 
ter 4, can be used to solve many counting problems. 


5 _ 

Recurrence Relations 

INTRODUCTION 

The number of bacteria in a colony doubles every hour. If a colony begins with five bac¬ 
teria. how many will be present in n hours? To solve this problem, let a» be the number 
of bacteria at the end of n hours. Since the number of bacteria doubles every hour, the 
relationship a„ =• 2it„-j holds whenever i? is a positive integer. This relationship, to¬ 
gether with the initial condition u t] - 5, uniquely determines for all nonnegative 
integers it, We can find a formula for a„ from this information. 

Some of the counting problems that cannot be solved using the techniques dis¬ 
cussed in Chapter 4 can be solved by finding relationships, called recurrence relations, 
between the terms of a sequence, as was done in the problem involving bacteria. We will 
study a variety of counting problems that can be modeled using recurrence relations. 
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EXAMPLE 1 


EXAMPLE 2 


We will develop methods in this section and in the following section for finding explicit 
formulae for the terms of sequences that satisfy certain types of recurrence relations. 


RECURRENCE RELATIONS 

In Chapter 3 we discussed how sequences can be defined recursively, Recall that a 
recursive definition of a sequence specifies one or more initial terms and a rule for 
determining subsequent terms from those that precede them. Recursive definitions can 
be used to solve counting problems. When they are, the rule for finding terms from 
those that precede them is called a recurrence relation. 


DEFINITION 1. A recurrence relation for the sequence \a n \ is an equation that 
expresses in terms of one or more of the previous terns of the sequence, namely, 
oo,a h .... fla-!, for all integers n with « «o> where no is a noanegative integer. A 

sequence is called a solution of a recurrence relation if its terms satisfy die recurrence 
relation. 


Let {<!„} be a sequence that satisfies the recurrence relation a„ - a„. | - > for/i = 

2, 3, 4— , and suppose that ao = 3 and a< = 5. What are at and ab¬ 
solution: We .see from the recurrence relation that a 2 = tq - ap = 5-3 =2 and 
aj = a; - = 2 — 5 = —3. ■ 


Determine whether tlie sequence {a„} is a solution of the recurrence relation a„ - 
2a„- [ - a „-2 for n - 2,3,4 .where a K - 3n for every nonnegative integer n. An¬ 

swer the same question where a n = 2" and where a„ = 5. 

Solution: Suppose that a n - 3 n for every nonnegative integer n. Then, for n > 2, we 
see that 2a„_| - a„_ 2 - 2[3<>i - I )J — 3(« - 2} = 3n = a„. Therefore, {«„}, where 
a„ = 3)j, is a solution of the recurrence relation. 

Suppose that a n - 2 n for every nonnegative integer n. Note that an = i, a-, =2, 
and <t 2 = 4, Since 2ai - «o -2-2-1 = 3 # aj, we see that {a„}, where a„ --- 2", is 
not a solution of the recurrence relation. 

Suppose that a n = 5 for every nonnegative integer n. Then for n at 2, wc see that 
a„ - 2a n -i — G/i -2 = 2-5-5- 5 = Therefore, {a ,where a n = 5, is a solution 
of the recurrence relation. ■ 


Tlie initial conditions for a sequence specify the terms that precede the first term 
where the recurrence relation takes effect. For instance, in Example I, tio = 3 and 
ti] - 5 are the initial conditions. The recurrence relation and initial conditions uniquely 
determine a sequence. This is the case since a recurrence relation, together with initial 
conditions, provide a recursive definition of the sequence. Any term of the sequence can 
be found from the initial conditions using the recurrence relation a sufficient number 
of times. However, there are better ways for computing tlie terms of certain classes of 
sequences defined by recurrence relations and initial conditions. We w ill discuss these 
methods in this section and in the following section. 
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EXAMPLE 3 


EXAMPLE 4 

web 


MODELING WITH RECURRENCE RELATIONS 

We can use recurrence relations to model a wide variety of problems, such as finding 
compound interest, counting rabbits on an island, determining the number of moves in 
the Tower of Hanoi puzzle, and counting bit strings with certain properties. 


Compound Interest Suppose that a person deposits $ 10,000 in a savings account at a 
bank yielding 11% per year with interest compounded annually. How much will be in 
the account after 30 y ears? 

Solution: To solve this problem, iet P„ denote the amount in die account after n years. 
Since the amount in the account after n years equals the amount in the account after 
n - 1 years plus interest for the nth year, we see that the sequence {P„l satisfies ihe 
recurrence relation 

P„ = F,.j + 0.11/Vi - (l.ll)F,-.,. 

The initial condition is P ( ] = 10.000. 

We can use an iterative approach to find a formula for F’„, Note that 

Pi = (Lll)Po 
p2 - (l-tl)F, - (1.11)% 

Pi ■ (l.ll)PJ = (1.11)% 

P n ■■= (Lli)F K .) = (l.ll)%. 

When we insert the initial condition P tl = 10, (XX), the formula P„ = (l.t I)" 10,00ft is 
obtained. We can use mathematical induction to establish its validity. That the formula 
is valid for n - 0 is a consequence of die initial condiiion. Now assume that P„ - 
(1. Il)*10,000. Then, from the recurrence relation and the induction hypothesis, 

p n ~\ = (i.ii)P n = (i.iixi.ur iaooo = (i.i if'*10,000. 

This shows that the explicit formula for P n is valid. 

Inserting n = 30 into the formula P n = (1.1 If 10,000 shows that after 30 years 
the account contains 

Pi B - (1.11 ) ?tl 10,000 = $228,922.97, ■ 


The next example shows how the population of rabbits on an island earn be modeled 
using a recurrence relation. 


Rabbits and the hibonacci Numbers Consider the following problem, which w as 
originally posed by Leonardo di Pisa, also known as Fibonacci, in the thirteenth cen¬ 
tury' in his book Liber abaci, A young pair of rabbits tone of each sex} is placed on 
an island. A pair of rabbits does not breed until they are 2 months old. After they are 
2 months old, each pair of rabbits produces another pair each month, as shown in 
Figure i. Find a recurrence relation for the number of pairs of rabbits on the island 
after n months, assuming that no rabbits ever die. 
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Month 

Reproducing | Young Total 

pairs j pairi | pairs 



t 

0 1 1 



2 

1 

0,1 ; 

i 


#* 

7 

i ! 

1 ! 1 j 2 

i 



4- 

■ 

1 2 , 2 



5 

| 

2 1 3 1 5 

#* #4ft 


6 

7 $ j 8 


«Mfc 


i 

Reproducing 

pairs 

You nr 
pairs 




FIG IRK, 1 Rabbits on an bland. 


Solution: Denote by L the number of pairs of rabbits after n months. We will show 

that f n , n - i, 2, 3.are the terms of the Fibonacci sequence. 

The rabbit population can be modeled using a recurrence rotation. At the end of 
the first month, the number of pairs of rabbits on the island is f\ = 1. Since this pair 
does not breed during the second month, fi - ) also. To find the number of pairs after 
n months, add the number on the island the previous month, 1 , and the number of 
newborn pairs, which equals /... ■>, since each newborn parr comes from a pair at least 
3 months old. 

Consequently, the sequence \f n ) satisfies the recurrence relation 
fn A-1 ^ /, : 

font > 3 together with the initial conditions f\ - land/; = 1. Since this recurrence 
relation and the in dial conditions unique. 1 )' determine this sequence, the number of pairs 
of rabbits on the island after n mouths is given by the nth Fibonacci number. ■ 


The nest example involves a gunoo.s puzzle. 

EXAMPLK 5 The Tower of Han-.*: A popular puzzle of the late nineteenth century, called the Tower 

of Hanoi, consists of three pegs mounted on a board together with disks of different 
. sizes. InitialSy these disks ate placed on the first peg in order of size, with the largest on 
the bottom (as show n in Figure 2). The rules of the puzzle allow disks to be moved one 
at a time from one peg to another as long as a disk is never placed on top of ,i smaller 
disk. The goal of the puzzle is to ha - c all the disks on the second peg in order of size, 
with the largest on the bottom. 

Let f!„ denote the number of moves needed to solve the Tower of Hanoi problem 
with n disks. Set up a recurrence relation for the sequence {//„}. 
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Solution: Begin with n disks on peg 1. We can transfer the top n - 1 disks, following 
the rules of the puzzle, to peg 3 using H„~\ moves (see Figure 3 for an illustration of 
the pegs and disks at this point). We keep the largest disk fixed during these moves. 
Then, we use one move to transfer the largest disk to the second peg. We can transfer 
the n - 1 disks on peg 3 to peg 2 using H n ..\ additional moves, placing them on top 
of the largest disk, which always stays fixed on the bottom of peg 2, Moreover, it is 
easy to see that the puzzle cannot be solved using fewer steps. This shows that 

H, = 2H„- t + 1. 

The initial condition is H\ = l, since one disk can be transferred from peg 1 to peg 2. 
according to the rules of the puzzle, in one move. 

We can use an iterative approach to solve this recurrence relation. Note that 

H„ = 2H n . j + 1 

= 2(2//.. : + I) + l = 2 2 M p; 3 + 2 t1 

- 2 '2//,. : • ! .' • 2 • i - 2 i //„-,! • : 2 - 2 - 1 

= 2" It + 2" “ ; 2 -i ‘ : 2 • 1 

s 2 ' : "' + 2"" 2 4 ■■■+2 4 1 

= 2" 1. 


_Peg_l___Peg 2 

FMIL'RE 3 An Intermediate Position in the Tower of Hanoi, 
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EXAMPLE 6 


We have used the recurrence relation repeatedly to express H„ in terms of previous 
terms of the sequence. In the next to last equality, the initial condition Hi - 1 has been 
used. The last equality is based on the formula for the sum of the terms of a geometric 
series, which can be found in Example 5 in Section 3,2. 

The iterative approach has produced the solution to the recurrence relation H n - 
2H n . i -t- 1 with the initial condition H\ = 1. This formula can be proved using math¬ 
ematical induction. This is left as an exercise for the reader at the end of the section. 

An ancient myth tells us that there is a tower in Hanoi where monks are transferring 
64 gold disks from one peg to another, according to the rules of the puzzle. They take 
1 second to move a disk. The myth says that the world will end when they finish the 
puzzle. How long after the monks started will the world end? 

From the explicit formula, the monks require 

2 W - 1 = 18.446,744,073.709, S31,615 

moves to transfer the disks. Making one move per second, it will take them more than 
500 billion years to solve the puzzle, so the world should survive a while longer than it 
already has. ■ 


Remark: Many people have studied variations of the original Tow er of Hanoi puzzle 
discussed in Example 5. Some variations use more pegs, some allow disks to be of the 
same size, and some restrict the types of allowable disk moves. One of the oldest and 
most interesting variations is the Reve’s puzzle,* proposed in 1907 by Henry Dudenev 
in his book The Canterbury■ Puzzles. The Reve’s puzzle involves pilgrims challenged 
by the Reve to move a stack of cheeses of varying sizes from the first of four stools 
to another stool without ever placing a cheese on one of smaller diameter. The Reve's 
puzzle, expressed in terms of pegs and disks, follows the same rules as the Tower of 
Hanoi puzzle, except that four pegs are used. You may find it surprising that no one has 
been able to establish the minimum number of moves required to solve this puzzle for 
n disks. However, there is a conjecture, now more than 50 years old, that the minimum 
number of moves required equals the number of moves used by an algorithm invented 
by Frame and Stewart in 1939. (See Exercises 48-55 at the end of this section and 
(St94] for more information,) 

Example 6 illustrates hnw recurrence relations can he used to count bit strings of a 
specified length that have a certain property. 


Find a recurrence relation and give initial conditions for the number of bit strings of 
length n that do not have two consecutive 0s, How many such bit strings are there nf 
length five? 


Solution: Let denote the number of bit strings of length n that do not have two 
consecutive 0s. To obtain a recurrence relation for {<z„}, note that by the sum rule, the 
number of bit strings of length n that do not have two consecutive 0s equals the number 
of such bit strings ending with a 0 plus the number of such bit strings ending with a 1. 
We will assume that n a 3, so that the bit string has at least three bits. 


*Reve, mors commonly spelled reeve, is an archaic word for governor. 
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EXAMPLE 7 


End with a 1 r 


Any hi; 'itnng of irrioih >1 - 1 uiifc 
rau twr Liinsetutive fJs 


End with a {): 


Any hit string of length n , with 
no two L’onsecutive ifs 


Number of bit .strings" 
of length n with no 
two consecutive C>s; 


Total: 

FIGLRE 4 Counting Bit Strings of Length n with No Two Consecutive Os, 


The bit strings of length n ending with I that do not have two consecutive Os are 
precisely the bit strings of length n - 1 with no two consecutive Os with a i added at 
the end Consequently, there are such bit strings. 

Bit strings of length n ending with a 0 that do no! have two consecutive Os must 
have 1 as their in - 1 tst bit; otherwise they would end with a pah of Os, It follows that 
the bit strings of length n ending with a 0 that have no two consecutive Os are precisely 
the bit strings of length n - 2 with no two consecutive Os with 10 added at the end. 
Consequently, there are a n .i such bit strings. 

We concJode, as illustrated in Figure 4. that 

a,, - a n | + <i„-2 
for ti £ 3 

t he initial conditions are a\ - 2, since both hit strings of length one, 0 and ! do 
not have consecutive Os. and at ■= 3. since the valid bit strings of length two are 01. 
10. and 11 To obtain at, we use the recurrence relation three times to find that 

to -- «2 + <ii = 3 + 2 - 5, 

04 " 0j '*■ u 2 — 3+3 — 8, 

05 - 04 + 03 = 8 + 5 13. B 


Remark: Note that {«, \ satisfies the same recurrence relation as the Fibonacci sequence. 
Since u = f; and = /■ it follows that a„ = /, +2 . 

Trie next example shows how a recurrence relation can be used to mode! the number 
of codewords that ate allowable using certain validity checks. 


( ndeword f numeration A computer system considers a string of decimal digits a 
valid codeword if it contains an even number of 0 digits. For instance, J 230407869 
ts vaiid, whereas 120987045608 is not valid. Let a„ be the number of valid n-digit 
codewords. Find a recurrence relation for a H , 

Solution: Note that - 9 since there are 10 one-digit strings, and only one, namely, 
the String 0, is not valid. A recurrence relation can be derived for this sequence by 
considering how a valid n-digit string can be obtained from strings ol it - 1 digits. 
There are two ways to form a valid string with n digits from a string with one fewer 
digit. 
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EXAMPLE 8 


web 


First, a valid string of /i digits can be obtained by appending a valid string of n - 1 
digits with a digit other than 0, This appending can be done in nine ways. Hence, a 
valid string with n digits can be formed in this manner in 9a n _i ways. 

Second, a valid string of n digits can be obtained by appending a 0 to a string of 
length n - 1 that is not valid. (This produces a string with an even number of 0 digits 
since the invalid string of length n - 1 has an odd number of 0 digits.) The number 
of ways that this can be done equals the number of invalid (n - l)-digit strings. Since 
there an? 10" _1 strings of length n - 1, and a n -\ are valid, there are lO "' 1 - valid 
n-digit strings obtained by appending an invalid string of length n - 1 with a 0 . 

Since all valid strings of length n are produced in one of these two ways, it follows 
that there are 

a n = 9a„_i + CIO" -1 - a„-]) 

= 80,-1 + Ilf 1 1 

valid strings of length n. ■ 

The next example establishes a recurrence relation that appears in many different 
contexts, 


Find a recurrence relation for C n , the number of ways to parenthesize the product of 
n + 1 numbers, xq ■ • x; * * ■ ■ ■ x n . to specify the order of multiplication. For example, 

C\ = 5 since there are live ways to parenthesize x 0 1 x 1 ■ x 2 ■ x 3 to determine the order of 
multiplication: (U 0 ■ x,)■ x 2 )-jc 3 , U'o ■ (a, ■ x 2 )) ■ .u, f x 0 - Xj) ■ (x 2 ■ xf), x 0 • t(x, ■ x : ) ■ ), 

and x 0 ■ (jtj ■ (ax • xf)), 

Solution: To develop a recurrence relation for C„, we note that however we insert 

parentheses in the product x 0 ■ j| ■ x 2 .x„, one operator remains outside all 

parentheses, namely, the operator for the final multiplication to be performed. [For 
example, in (,r 0 • (x, * x 3 )) ■ x 3 , it is the final while in (jr 0 • xf) ■ fax x-i) it is 
Ihe second “■”] This final operator appears between two of the n + 1 numbers, say. x k 
and x ;, + 1 . There are C(-C„ k .. ] ways to insert parentheses to determine the order of the 
n + I numbers to be multiplied when the final operator appears between x k and x;. ( , 

since there are C k ways to insert parentheses in the product x 0 • x Y . x k to determine 

the order in which these k + I numbers are be be multiplied and C*_ *_ t ways to insert 

parentheses in the product xtn * x k+2 .x„ to determine the order in which these 

n - k numbers are to be multiplied, Since this final operator can appear between any 
two of the n f 1 numbers, it follows that 

C„ - CqC k -} +C[C „-2 + '''•*■ C^-aCj + C„-jCt) 

FI-J 

- V ( a :• 
t-o 

Note that the initial conditions are Co = 1 and Cj — 1. This recurrence relation can 
he solved using the method of generating functions, which will be discussed in Section 
5.4, It can be shown that C„ ~ C(2n, n)l(n + 1 ). (See Exercise 41 at the end of that 
section,) g 

The sequence {C„} is the sequence of Catalan numbers. This sequence appears as 
the solution of many different counting problems besides the one considered here (see 
the chapter on Catalan numbers in [MiRoSU] or [Rob84| for details). 
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Exercises 


1. Find the first live terms of the sequence defined by each 
of the following recurrence relations and initial condi¬ 
tions, 

a) a„ 6(3„~i . rid - 2 

b) n„ = a*. lt d| ^2 

C) ti„ = <J„ ; I 3= 1, a ( = 2 

d) £i„ = III!,, ! f 2,1*0 = l.£/| = 1 

ei u„ - £<„. i -t- a* j, an = Fa, = 2,ai = 0 

2. Show that the sequence {«„} is a solution of the recur¬ 
rence relation u n - -3a„-\ + 4ri„_2 if 

ai ei,, -■ 0. 

bl <)„ ■' 1. 

e) a„ - (-4)*. 

d) a„ = 2(-4r - 3, 

3. Is the sequence {a K ) a solution of the recurrence relation 

= X„\. | - I r>/;. ; if 

a) a,, = 03 

b) u„ = 1? 

C) a„ = 2"? 

d) a u = 4 n ? 

e) «, = rt4'* ? 

fl a„ = 2-4" l 3«4"? 
gl «» = ( ;4)"3 
h) a„ - ft : 4“? 

4. For each of the following sequences find a recurrence 
relation satisfied by this sequence, (The answers are not 
unique since there are infinitely many different recur¬ 
rence relations satisfied by any sequence. ) 

a! a„ 3 
b) n„ = 2n 
d a„ ■■■ 2/t ^ 3 
d) a,, - 3" 
ei a„ - tv 

f) a„ n' F n 

g) a„ n v (- I )" 

h) a„ - n! 

5. Find the sol ution to each of the following recurrence re¬ 
lations and initial conditions. Use an iterative approach 
such as that used in Example 5. 

a) a„ ••= 3i/„ i, un - 2 

lit fi„ - ei,,. | -+ 2, a ii - 3 

C) A, ■ O n . | + /I, 11:) - 1 

d) a„ ■ + 2n *- 3. a;, = 4 

e) a„ = 2a„ -i - 1, a 0 - 1 

f) (l„ - 3(1,,-1 4 l,B|| - 1 
ft) a„ = Wi„ i, no ■- 5 

h) a n - 2m;„ i, «u = 1 

b- A person deposits $ IOOO in an account that yields 9% 
interest compounded yearly. 

ai Set up a recurrence relation for the amount in the 
account at the end of n years. 


h) Find an explicit formula for she amount in the ac¬ 
count at the end of n years, 

c) How much money will the account contain after 100 

years? 

7, Suppose that the number of bacteria in a colony triples 
every hour. 

a) Set up a recurrence relation for the number of bac¬ 
teria after n hours have elapsed, 

b) If 100 bacteria are used to begin a new colony, how 
many bacteria will be in the colony in 10 hours? 

8, Assume that the population of the world in 1999 is 6 
billion and is growing at the rate of 1,3% a year. 

a) Set up a recurrence relation for the population of the 
world n years after 1999, 

b) Find an explicit formula for the population of the 
world 7i years after 1999. 

c) What will the population of the world be in 2020? 

9, A factory makes custom sports cars at an increasing 
rate, In the first month only one car is made, in the sec¬ 
ond month two cars are made, and so on, with n cars 
made in the nth month. 

a) Set up a recurrence relation for the number of cars 
produced in the first n months by this factory, 

b) How many cars are produced in the first year? 

c) Find an explicit formula for the number of cars pro¬ 
duced in the first n months by this factory. 

10. An employee joined a company in 1987 with a starti ng 
salary' of $50,000. Every year this employee receives 
a raise of $1000 plus 5% of the salary of the previous 
year, 

a) Set up a recurrence relation for the salary of this 
employee n years after 1987, 

b) Whal is the salary of this employee in 1995? 

c) Find an explicit formula for the salary of this em¬ 
ployee n years after 1987. 

11. Use mathematical induction to verify the formula de¬ 
rived in Example 5 for the number of moves required 
to complete the Tower of Hanoi puzzle. 

12. a) Find a recurrence relation for the number of permu¬ 

tations of a set with n elements, 
b) Use this recurrence relation to find the number of 
permutations of a set with n elements using itera¬ 
tion. 

13. A vending machine dispensing books of stamps ac¬ 
cepts only Susan B. Anthony dollar coins, $1 bills, and 
$5 bills. 

a) Find a recurrence relation for the number of ways 
to deposit /I dollars in the vending machine, where 
the order in w hich the coins and bills are deposited 
matters. 

b) What are the initial conditions? 
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cl How many ways are there to deposit $10 for a book 
of stamps? 

14. A country uses as currency coins with values of 1 peso, 
2 pesos, 5 pesos, anil 10 pesos and bills with values of 
5 pesos, 10 pesos. 20 pesos, 50 pesos, and 100 pesos. 
Find a recurrence relation for the number of ways to 
pay a bill of n pesos if the order in which the coins and 
bills are paid matters, 

15. How many ways are there to pay a bill of 17 pesos usi ng 
the currency described in Exercise 14, where the order 
in which coins and bills are paid matters'? 

*16, a) Find a recurrence relation for the number of strictly 
increasing sequences of positive integers that have 
1 as their first term and n as their last term where 
n is a positive integer. That is, sequences tj], a->, 
.... flj where, a \ = l.rij = n, and a,- < ci t j for 
j = 1,2__ A- — l. 

b) What ttre the initial conditions? 

c) How many sequences of the type described in (a) 
are there when n is a positive integer with n Hr 2? 

17, a) Find a recurrence relation for the number of hit 
strings of length n that contain a pair of consecu* 
live 0s. 

It) What are the initial conditions? 
c) How many bit strings of length seven contain two 
consecutive Os? 

IS. a) Find a recurrence relation for the number of bit 
strings of length n that contain three consecutive 0s. 
b) What are the initial conditions? 
el How many bit strings of length seven contain three 
consecutive 0s? 

19. a) Find a recurrence relation for the number of bit 
strings of length n that do not contain three consec¬ 
utive 0s. 

b) What are the initial conditions'.' 

c) How many bit strings of length seven do not contain 
tliree consecutive Os? 

*20. a) Find a recurrence relation for the number of bit 
strings that contain the string (11. 
b! What are the initial conditions? 
c) How many bit strings of length seven contain the 
string 01 7 

21. a t Find a recurrence relation for the number of ways to 

climb n stairs if tire person climbing the stairs can 
take one stair or two stairs at a time. 

b) What are the initial conditions? 

e) How many ways can this person climb a flight of 
eight stairs'? 

22 . a) Find a recurrence relation for the number of ways to 

climb « stairs if the person climbing the stairs can 
take one, two, or three stairs at a time. 
h) What are the initial conditions? 

c) How many ways can this person climb a flight of 
eight stairs? 


A string that contains only 0s, 1 s, and 2s is called a ternary 
string. 

23. a) Find a recurrence relation for the number of ternary 

strings that, do not contain two consecutive 0s 

b) What are the initial conditions? 

c) How many ternary strings of length six do not con¬ 
tain two consecutive Os? 

24, a) Find a recurrence relation for the number of ternary 

strings that contain two consecutive 0s. 

b) What are the initial conditions? 

c) How many ternary strings of length six contain two 
consecutive 0s? 

*25, a) Find a recurrence relation for the number of ternary 
strings that do not contain two consecutive 0s or two 
consecutive Is, 

b) What are the initial conditions? 
e) How many ternary strings of length six do not con¬ 
tain two consecutive 0s or two consecutive 1 s? 

*26, a) Find a recurrence relation for the number of ternary 
strings that contain either two consecutive 0s or tw o 
consecutive Is. 

b) What are the initial conditions? 
e) How many ternary strings of length six contain two 
consecutive Os or two consecutive Is? 

*27. a) Find a recurrence relation for the number of ternary 
strings that do not contain consecutive symbols 
which are the same, 

b) What are the initial conditions? 

c) How many ternary' strings of length six do not con¬ 
tain consecutive symbols which are the same? 

**28. a) Find a recurrence relation for the number of ternary 
strings that contain two consecutive symbols which 
are the same. 

b) What are the initial conditions? 

c) How many ternary strings of length six contain con¬ 
secutive symbols that are the same? 

29. Messages are transmitted over a communications chan¬ 
nel using two signals. The transmittal of one signal re¬ 
quires 1 microsecond, and the transmittal of the other 
signal requires 2 microseconds, 

a) Find a recurrence relation for the number of differ¬ 
ent messages consisting of sequences of these two 
signals, where each signal in the message is imme¬ 
diately followed by the next signal, that can be sent 
in n microseconds. 

b) What are the initial conditions? 

c) How many different messages can be sent in 10 mi¬ 
croseconds using these two signals? 

30. A bus driver pays all tolls, using only nickels and 
dimes, by throwing one coin at a time into the mechan¬ 
ical toll collector. 

a) Find a recurrence relation for the number of differ¬ 
ent ways the bus driver can pay a toll of n cents 
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( where the order in which the coins are used mat¬ 
ters'). 

b) In how many different ways can the driver pay a 
mil (if 45 cents? 

31. a) Find the recurrence relation satisfied, by R„, where 
R„ is the number of regions that a plane is divided 
into by n lines, if no two of the lines are paral¬ 
lel and no three of the lines go through the same 
point. 

b) Find R„ using iteration, 

*32. a) Find the recurrence relation satisfied by R„, where 
F:„ is the number of regions into which the sur¬ 
face of a sphere is divided into by n great cir¬ 
cles (which are the intersections of the sphere and 
planes passing through the center of the sphere), if 
no three of the great circles go through the same 
point. 

hi Find R„ using iteration. 

*33. aj Find the recurrence relation satisfied by S„, where 
$„ is the number of regions into which three- 
dimensional space is divided into by n planes if ev¬ 
ery' three of the planes meet in one point, but no four 
of the planes go through the same point, 
b) Find S„ using iteration. 

34. Find a recurrence relation for the number of bit se¬ 
quences of length n w ith an even number of Os. 

35. How- many bit sequences of length seven contain an 
even number of Os? 

36. a) Find a recurrence relation for the number of ways 

to completely cover a. 2 x n chessboard with I x 1 
dominos. {Hint. Consider separately the coverings 
where the position in the top right corner of the 
chessboard is covered by a domino positioned hor¬ 
izontally and where it is covered by a domino posi¬ 
tioned vertically.) 

b) What are the initial conditions for the recurrence 
relation in part (aI? 

c) How many ways are there to completely cover a 
2 X 17 chessboard with I x 2 dominos? 

37. a) Find a recurrence relation for the number of ways to 

lay out a walkway with slate tiles if the tiles are red, 
green, or gray, so that no two red tiles are adjacent 
and tiles of the same color arc considered indistin¬ 
guishable. 

b) What are the initial conditions for the recurrence 
relation in part (aj? 

c) How many ways arc there to lay out a path of seven 
tiles as described in pail (a)? 

38. Show that the Fibonacci numbers satisfy the recurrence 

relation /„ - 5/„_ 4 - 3 f„ s for n = 5, ft, 7_ to¬ 

gether with the initial conditions/® - 0 , f -- 1 . fa = 

'■/>. - 2, and /i - 3. Use this recurrence relation to 
show that / Sll is divisible by 5, for n = t, 2, 3,... . 

*39. Let Sun. m denote the number of onto functions from 
a set wdli m elements to n set with b elements. Show 


that S(m, n) satisfies the recurrence relation 

"“-1 

S(m, n ) ■■■ n m ~ ^ C(n, k)S{m, k) 
i= I 

whenever m s « and n > 1 , with the initial condition 
S(m 1 ) = 1 . 

40. a) Write out all the ways the product ■ x\ ■ x; 

aj, • can be parenthesized to determine the order 
of multiplication. 

b) Use the recurrence relation developed in Example 
8 to calculate Q, the number of ways to parenthe¬ 
size the product of five numbers so as to determine 
the Order of multiplication. Verify that you listed (he 
correct number of w ays in part (a). 

c) Check your result in part (b) by finding C 4 . using 
the closed formula for C„ mentioned in the solution 
of Example 8 . 

41. a) Use the recurrence relation developed in Example 8 

to determine C 5 , the number of ways to parenthesize 
the product of six numbers so as to determine the 
order of multiplication, 

b) Check your result with the closed formula for (C 
mentioned in the solution of Example 8 . 

*42. In the Tower of Hanoi puzzle, suppose our goal is to 
transfer all n disks from peg 1 to peg 3, but we cannot 
move a disk directly between pegs I and 3, Each move 
of a disk must be a move involving peg 2. As usual, we 
cannot place a disk on top of a smaller disk, 

a) Find a recurrence relation far the number of moves 
required to solve the puzzle for n disks with this 
added restriction. 

b) Solve this recurrence relation to find a formula for 
the number of moves required to solve the puzzle 
for ft disks. 

c) How many different arrangements are there of the 
n disks on three pegs so that no disk is on top of a 
smaller disk? 

d) Show that every allowable arrangement of the n 
disks occurs in the solution of this variation of the 
puzzle. 

Exercises 43—47 deal with a variation of the Josephus 
problem described by Graham, Ktiuth, and Patashnik in 
[GrKnPa94j, This problem is based on an account by the 
historian Flauvius Josephus, who was part of a hand of 41 
Jewish rebels trapped in a cave by the Romans during the 
Jewish-Roman war of the first century. The rebels preferred 
suicide (0 capture; they decided to form a circle and to re¬ 
peatedly count off around the circle, killing every third rebel 
left alive. However, Josephus and another rebel did not want 
to be killed this way; they determined the positions where 
they should stand to be the last two rebels remaining alive. 
The variation we consider begins with n people, numbered 
1 ton, standing around a circle. In each stage, every second 
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person still left alive is eliminated until only one survives. 
We denote the number of the survivor by /(nl, 

43. Determine the value of J(n) for each integer n with 
1 -s n < 16. 

44. Use the values you found in Exercise 43 to conjecture 
a formula for J(n). (Hint: Write n - 2 m + k, where m 
is a nonnegative integer and k is a nonnegative integer 
less than 2™.) 

45. Show that J(n) satisfies the recurrence relation 
J(2n) = 2J(n) - \ and 7(2n + 1) = U(n) + 1, for 
n c; 1, and /(11 - 1. 

46. Use mathematical induction to prove the formula you 
conjectured in Exercise 44, making use of the recur¬ 
rence relation from Exercise 45. 

47. Determine /(I(XU, 7(1000), and 7(10000} front your 
formula for Jin). 

Exercises 48-55 involve the Reve’s puzzle, the variation 
of the Tower nf Hanoi puzzle with four pegs and n disks. 
Before presenting these exercises, we describe the Frame- 
Stewart algorithm for moving the disks from peg 1 to peg 
4 so that no disk is ever on top of a smaller one. This al¬ 
gorithm, given the number of disks n as input, depends on 
a choice of an integer k with I s k. £ n. When there is 
only one disk, move it front peg 1 to peg 4 and stop. For 
n > 1, the algorithm proceeds recursively, using the fol¬ 
lowing three steps Recursively move the stack of the n - k 
smallest disks from peg 1 to peg 2. using all four pegs. Next 
move the stack of the k largest disks from peg 1 to peg 
4, using the three-peg algorithm from the Tower of Hanoi 
puzzle without using the peg holding the n - k smallest 
disks, Finally, recursively move the smallest n - k disks 
to peg 4. using all four pegs. Frame and Stewart showed 
that to produce the fewest moves using their algorithm, k 
should be chosen to be the smallest integer such that n 
does not exceed 4 = kik + l)/2, the Ath triangular num¬ 
ber, i.e., 4 1 < rt s; 4 . The unsettled conjecture, known as 
Frame’s conjecture, is that this algorithm uses the fewest 
number of moves required to solve Lite puzzle, no matter 
how the disks are moved. 

48. Show that the Reve's puzzle with three disks can be 
solved using five, and no fewer, moves. 


49. Show that the Reve’s puzzle with four disks can be 
Solved using nine, and no fewer, moves, 

50. Describe die moves made by the Frame-Stewart al¬ 
gorithm, with A chosen so that the fewest moves are 
required, for 

a) 5 disks, b) 6 disks, c) 7 disks, d) 8 disks. 

*51. Show that if R(n) is the number of moves used by the 
Frame-Stewart algorithm to solve the Reve’s puzzle 
with n disks, where A is chosen to be the smallest inte¬ 
ger with n £ k(k + 1 )/2, then R(n) satisfies the recur¬ 
rence relation with Rin) = 2Rin - k) 4- 2* - 1, with 
ff(0"J = 0 and Ri 1) = 1. 

*52. Show that if A is as chosen in Exercise 51, then Rin) - 
R(n - 1) = 2*T 

*53, Show that if A is as chosen in Exercise 51, then lf(n) = 

I2 1 - 1 - (f* - n)2 A -'. 

*54. Use Exercise 53 to give an upper bound on the num¬ 
ber of moves required to solve the Reve's puzzle for all 
integers n with I £ n £ 25. 

*55. Show that R(n) is (X v n?. ■ • 

Let {«„} be a sequence of real numbers. The backward dif¬ 
ferences of this sequence are defined recursively as follows. 

The first difference Va„ is 

Va„ = a„ - a n -\. 

The (A- +■ 1 )th difference V k 11 a„ is obtained from by 
V ,:+1 u n = V k a„ - VVi. 

56. Find Va„ for the sequence {a„ ]■ where 

a) a n = 4. b) a,, = 2n. c) «„ = rr. d) a„ ~ 2”. 

57, Find Vfo„ for the sequences in Exercise 34. 

58. Show that a M -.j = a„ - Va„. 

59, Show 1 that a n .. 2 = a„ - 2Va„ + V 2 ri„. 

*60. Prove that £!„_* can be expressed in terms of 
a n , Vo*, V 2 a n ,.... V k a„. 

61. Express the recurrence relation a„ - a„ , + a„_; in 
terms of a„, Va K , and V z a„. 

62. Show that any recurrence relation for the sequence {a„} 

can be w ritten in terms of a K , Vu,, V 2 a„.The re¬ 

sulting equation involving the sequences and its differ¬ 
ences is called a difference equation. 


5. 2 

Solving Recurrence Relations 

INTRODUCTION 


A wide variety of recurrence relations occur in models. Some of these recurrence rela¬ 
tions can be solved using iteration or some other ad hoc technique. However, erne im¬ 
portant class of recurrence relations can be explicitly solved in a systematic way. These 
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EXAMPLE 1 


EXAMPLE 2 


are recurrence relations that express the terms of a sequence as linear combinations of 
previous terms. 


Dflroffl'KJS l..-. A linear hnmogemms ncurretm h vttih 

■ - A, i> ‘s:U-.i4 'i-i. V -'-■■■ ' ‘ 4f»* )♦ ft ,' 

«■» * ^%-t + $«i.-a + • + s , r _ 


where ci,C 2 , ..., are real numbers, and c* # 0, 


The recurrence relation in the definition is linear since the right-hand side is a sum 
of multiples of the previous terms of the sequence. The recurrence relation is homo¬ 
geneous since no terms occur that are not multiples of the ajs. The coefficients of the 
terms of the sequence are all constants, rather than functions that depend on n. The 
degree is k because a„ is expressed in tenns of the previous k terms of the sequence. 

A consequence of the second principle of mathematical induction is that a sequence 
satisfying the recurrence relation in the definition is uniquely determined by this recur¬ 
rence relation and the k initial conditions 

a 0 = Co.£J| - C\ . a k .[ - C,t_j- 


The recurrence relation P H = (1.1I)P ([ _. T is a linear homogeneous recurrence relation 
of degree one. The recurrence relation f„ - f a l + f n . 2 is a linear homogeneous re¬ 
currence relation of degree two. The recurrence relation a„ = a„- 5 is a linear homoge¬ 
neous recurrence relation of degree five. ■ 


Some examples of recurrence relations that are not linear homogeneous recurrence 
relations with constant coefficients follow. 


The recurrence relation ] + a*_ 2 is not linear. The recurrence relation H„ = 

+ 1 is not homogeneous. The recurrence relation B„ - nB, : l does not have 
constant coefficients. g 


Linear homogeneous recurrence relations are studied for two reasons. First, they 
often occur in model ing of problems. Second, they can be systematical ly solved. 

SOLVING LINEAR HOMOGENEOUS RECURRENCE RELATIONS 
WITH CONSTANT COEFFICIENTS 


The basic approach for solving linear homogeneous recurrence relations is to look for 
solutions of the form a n - r n , where t is a constant. Note that a„ = r* is a solution of 
the recurrence relation u n = C\(i„ i + cja n j • + Cf,a n -k if and only if 

r" = C [r ,! £ -I- Qr" M ■■■ -r a-r r ‘~ k . 

When both sides of this equation are divided by r J, ~ k and the right-hand side is sub¬ 
tracted from the left, we obtain the equivalent equation 

J _ r *-1 „ Jk-2 „ „ 

fir -c 2 r --Cjj -1 r - c* = 0. 
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THEOREM 1 


Consequently, the sequence {«„} with a„ - r" is a solution if and only if r is a solu¬ 
tion of this last equation, which is called the characteristic equation of the recurrence 
relation, The solutions of this equation tire called the characteristic roots of the recur¬ 
rence relation. As we will see, these characteristic roots can be used to give an explicit 
formula for all the solutions of the recurrence relation. 

We will first develop results that deal with linear homogeneous recurrence relations 
with constant coefficients of degree two. Then corresponding general results when the 
degree may be greater than two will he stated. Because the proofs needed to establish 
the results in the general case are more complicated, they will not be given in the text. 

We now turn our attention to linear homogeneous recurrence relations of degree 
two. First, consider the ease when there are two distinct characteristic roots. 

Let ci and c 2 be real numbers. Suppose that r 2 - c\r -c 2 - 0 has two distinct 
roots T\ and r 2 . Then the sequence {a„} is a solution of the recurrence relation a„ - 
iTfl«-t + cjfln-r if and only if a„ = aj rf + for n = 0,1,2,, where <* t and 
«2 are constants. 

Proof: We must do two things to prove the theorem. First, it must be shown that if r, 
and r> are the roots of the characteristic equation, and a, and a 2 are constants, then the 
sequence {a„} with a B - a 1 tf + airg i s 11 solution of the recurrence relation. Second, it 
must be shown that if the sequence is a solution, then a„ = a, 4 a ir 1 -! for some 

constants aq and u' 2 . 

Now we will show that if a„ = a 1 rf a 2 Pf then the sequence {«.,} is a solution 
of the recurrence relation. Since tq and r> are roots of r - C|f- t.n = 0. it follows that 
ri - -I- t’2, r| = c'lfq + c 2 . 

From these equations, we see that 

+ c 2 a, t -2 - i‘i («[/■" 1 + a 2 r| 1 ) + Qfqqr " -2 -t- a 2 rf ") 

“ «! 2 U‘[r ( + c 2 ) 4 a 2 r” “(t'i r 2 4 03 ) 

= a 1 r, rj - air" ~ri 
= a i r" 4 (l ri 
= a n . 

This shows that the sequence {«„} with = a L rj‘ + uir'i is a solution of the recurrence 
relation. 

To show that every solution {«„} of the recurrence relation a„ - c|U rt .| + Citi,,-; 
has a„ = oqrj 1 + a 2 rf for n - 0, 1, 2,, for some constants e*| and a 2 > suppose that 
{« n } is a solution of the recurrence relation, and the initial conditions a 0 — Cq and 
ci\ = C] hold. It will be shown that there are constants a [ and 02 so that the sequence 
{u n } with a„ = «] r” 4 a 2 r% satisfies these same initial conditions. This requires that 

(fij = Co ~ «i 4£r-j, 
ft] — C] = ct 1 r] * a 2 r 2 . 

We can solve these two equations for oq and as. From the first equation it follows that 
= (<> ~ ft]. Inserting this expression into the second equation gives 

C] - a | f, 4 (C 0 - K|)p2. 

Hence. 

C[ = atifn - r 2 ) - 1 - Can. 
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EXAMPLE 3 


EXAMPLE 4 


This shows that 

(A - Carj) 


and 


a 2 = C (J “ a | 


~ A - 


(Cl - Qjri) 

n - n 


Con - A 

r\ - n 


where these expressions for a , and a 2 depend on the fact that n ¥= r 2 , (When r ( - r 2 , 
this theorem is not true.) Hence, with these values for a i and a 2 , the sequence {a,} with 
<* i tj + £* 2*2 satisfies the two initial conditions. Since this recurrence relation and these 
initial conditions uniquely determine the sequence, it follows that a n - airf + ayiS- 

□ 


The characteristic roots of a linear homogeneous recurrence relation with constant 
coefficients may be complex numbers. Theorem 1 (and also subsequent theorems in 
this section) still applies in this case. Recurrence relations with complex characteristic 
roots will not be discussed in the text. Readers familiar with complex numbers may 
wish to solve Exercises 38 and 39 at the end of this section. 

The following examples illustrate the usefulness of the explicit formula given in 
Theorem 1. 


mat is the solution of the recurrence relation 
= a„. i + 2a„-2 

with ao - 2 and a t - 7? 

Solution: Theorem l can be used to solve this problem. The characteristic equation of 
the recurrence relation is r 2 - r - 2 = 0. Its roots are r = 7 and r = - 1. Hence, the 
sequence {o„} is a solution to the recurrence relation if and only if 

u„ = a,2 n + a 2 (-l) n , 

for some constants a\ and cri. From the initial conditions, it follows that 

op = 2 = a i + ofj, 

fli = 7 = ai ■ 2 + ct2 * {— 1). 

Solving these two equations shows that ci] - 3 and ay - -1. Hence, the solution to 
the recurrence relation and initial conditions is the sequence la,,} with 

a„ = 3-2" - (-1}". ■ 


Find an explicit formula for the Fibonacci numbers. 

Solution: Recall that the sequence of Fibonacci numbers satisfies the recurrence re¬ 
lation f„ - /„. | + f„_ 2 and also satisfies the initial conditions f 0 - 0 and f = 
i. The roots of the characteristic equation r 2 - r - 1 = 0 are r| ~ (1 + v 5)/2 and 
ra - (1 - v'5)/2. Therefore, from Theorem 1 it follows that the Fibonacci numbers 
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THEOREM 2 


EXAMPLE 5 


are given by 


fn = «1 


1 + v'5 



for some constants a\ and « 2 - The initial conditions /q = 0 and f\ = I can be used to 
find these constants. We have 


fo - ai + a 2 - 0. 


/i 




= 1. 


The solution to these simultaneous equations for a i and a 2 is 
a j = l/v'5, a 2 = - l/v'5. 

Consequently, the Fibonacci numbers are given by 


A + v^T i ( 

r\ n 

1 - v ; 5 \ 



T 

■ 


Theorem 1 does not apply when there is one characteristic root of multiplicity two. 
This case can be handled using the following theorem. 

Let ct and c 2 be real numbers with c 2 ? 0. Suppose that r 2 - c\r-c-i = Ohas only 
one root r«. A sequence {e R } is a solution of the Tecurreoeerektion a n - c-\a n ~ \ + 
C 2 a „-2 if and only if a„ = + &&&$, for n = 0,1,2,, where oq and a 2 are 

constants. 

The proof of Theorem 2 is left as an exercise at the end of the section. The following 
example illustrates the use of this theorem. 


What is the solution of the recurrence relation 
£J„ = 6£t„-i - 9fl„-2 

with initial conditions ao = 1 and aj ~ 6? 

Solution: The only root of r 1 “ 6r + 9 = 0 is r = 3. Hence, the solution to this recur¬ 
rence relation is 

= ttl 3" + a 2 ii3* 

for some constants a ( and ov- Using the initial conditions, it follows that 
% = 1 = a], 

Oi = fi = ■ 3 + a 2 ■ 3. 

Solving these two equations shows that«] = 1 anda 2 - 1. Consequently, the solution 
to this recurrence relation and the initial conditions is 

a„ = 3" + n3 n , • 
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THEOREM 3 


EXAMPLE 6 


We will now state the general result about the solution of linear homogeneous re¬ 
currence relations with constant coefficients, where the degree may be greater than two, 
under the assumption that the characteristic equation has distinct roots. The proof of this 
result will he left as an exercise for the reader. 

Let cj, C 2 ,.... c t be real numbers. Suppose that the characteristic equation 

r k - c k = O 

has k distinct roots rj, r 2 ,..., r k . Then a sequence {o„} is a solution of the recurrence 
relation 

On - i + t’2«n-2 + -■’ + c t a n -i 

if and only if 

On = «iff + 02^2 + ■" + a*r£ 
for n = 0,1.2,..., where a\, aj ,..., a* are constants. 

We illustrate the use of the theorem with an example. 


Find the solution to the recurrence relation 
a„ - 6w n -j _ lla„-2 + 6 a„ y 
with the initial conditions fly = 2 ,a\ - 5, andaj • 15. 

Solution: The characteristic polynomial of this recurrence relation is 
r'-br + Hr- 6 . 

The characteristic roots are r = 1, r = 2. and r = 3, since r 1 - hr 2 + 11 r - 6 = 
l.r - l)(r - 2)(r - 3). Hence, the solutions to this recurrence relation are of the form 

a„ ~ a ( ■ 1" + a 2 ■ 2" + - 3". 

To find the constants aj, an, and ay, use the initial conditions. This gives 
tty - 2 = ai + a 2 + ay, 
a i — 5 = aj +■ ai ■ 2 + <ij -3, 
a 2 = 15 = a | + oi ■ 4 + nr 3 ■ 9. 

When these three simultaneous equations are solved for a j, and ct^, wc find that 
ai = l.cra = -l.andtti = 2. Hence, the unique solution to this recurrence relation 
and the given initial conditions is the sequence {«„} with 

a„ - 1 - 2 n - 2 ■ 3". ■ 


We now state the most genera) resul! about linear homogeneous recurrence relations 
with constant coefficients, allowing the characteristic equation to have multiple roots. 
The key point is that for each rooi r of the characteristic equation, the general solution 
has a summand of the form P(n)r n , where P(n) is a polynomial of degree m - 1. with m 
the multiplicity of this root. We leave the proof of this result as a challenging exercise 
for the reader. 
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EXAMPLE 7 


EXAMPLE 8 


Let a, C 2 ,...,Cibe teal numbers, Suppose that the characteristic equation 
r 1 - Cjr* -1 -- c t =*= 0 

has t distinct mote t\, r%...,r t with multiplicities mi, mi,..., m t , respectively, so 
that Wj 5 1 for i - 1,2,,,., t and mi + m 2 + - ■ ■ + m ( = it. Then a sequence {a„) 
is a solution oftherecirrietice relation 

fl n = T C2«n-2 + '' ‘ + C*C B -t 

if and only if 

a n = («[,o + «u n -*■ 

+ + ana + ■•■ 

+ — + (or/,e + a,.in + 4 

for n = 0,1,2,..where ai j are constants for 1 := i A t and 0 <, j < m* - L 

The following example illustrates how Theorem 4 is used to find the general form of 
a solution of a linear homogeneous recurrence relation when the characteristic equation 
has several repeated roots. 


Suppose that the roots of the characteristic equation of a linear homogeneous recurrence 
relation are 2, 2, 2, 5,5, and 9 (that is, there are three roots, the root 2 with multiplicity 
three, the root 5 w ith multiplicity two, and the root 9 with multiplicity one). What is the 
form of the general solution? 

Solution: By Theorem 4, the general form of the solution is 

(«]_!) + Q-j j n 4 a'utr)2" - (rr 2 ,o + osjjiW + di(|f, ■ 


We now illustrate the use of Theorem 4 to solve a linear homogeneous recurrence 
relation with constant coefficients w hen the characteristic equation has a root of multi¬ 
plicity three. 


Find the solution to the recurrence relation 
a n — -3h b .-j —3 a„-2 j 

with initial conditions «o = l.n, = -2. and a > - 1. 

Solution: The characteristic equation of this recurrence relation is 
r* 4- 3r- 4 3r 4 1 - 0. 

Since r 3 t 3 r + 3r 4 1 - (r 4 1) J , there is a single root r = -1 of multiplicity three 
of the characteristic equation. By Theorem 4 the solutions of this recurrence relation 
are of the form 

ii n -- OM 0 ( "IF 4 4a 0 n 2 (-iy. 
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EXAMPLE 9 


THEOREM 5 


To find the constants a | $, a L i and oti i, use the initial conditions. This gives 
flo = 1 = a i i0 , 

«l = -2 - -aj o - nri.i - aq.j, 
az — - l - ai,o 4 2au + 4«u, 

The simultaneous solution of these three equations js = 1, oi ( = 3, and a < l2 
2. Hence, the unique solution to this recurrence relation and the given initial condi¬ 
tions is the sequence {a„} with 

= (1 + in - 2n 2 )(-1 )T g 

LINEAR NONHOMOGENEOUS RECURRENCE RELATIONS 
WITH CONSTANT COEFFICIENTS 

We have seen how to solve linear homogeneous recurrence relations with constant coef¬ 
ficients. Is there a relatively simple technique for solving a linear, but not homogeneous, 
recurrence relation with constant coefficients, such as a„ = 3 a„~i + 2«'? We will see 
that the answer is yes for certain families of such recurrence relations. 

The recurrence relation a„ - 3a„.. i + 2 n is an example of a linear nonhomofio 
neous recurrence relation with constant coefficients, that is, a recurrence relation of 
the form 

a n - ci« n -i + c 2 a a - 2 + + t\a n .. k + Fin). 

where c\,cz,.... c t are real numbers and Fin) is a function not identically zero de¬ 
pending only on n. The recurrence relation 

mi = -r cjs„. 2 + ■ ■ ■ +- Cf.a n -t 

is called the associated homogeneous recurrence relation. It plays an important role 
in the solution of the nonh o mo ge neous recurrence relation. 

Each of the recurrence relations a„ - a n -i + 2 rt , a fl = a n -\ + a n ..z + >r 4 n 4 |. 
a„ = 3o,,_ | +rt3 n . and a„ = 4 a f .;4a fl -s + n! is a linear nonhomoge neous recur¬ 

rence relation with constant coefficients. The associated linear homogeneous recurrence 
relations area,, = a n - h a n = a„ i+a„ z, a„ = 3a n _i,anda„ = m-i+m- : + m- 3 , 

respectively. g 


The key fact about linear nonhomogeneous recurrence relations with constant 
coefficients is that every solution is the sum of a particular solution and a solution 
of the associated linear homogeneous recurrence relation, as the following theorem 
shows. 

MaT} is a particular solution of the aonhomogeneous linear recurrence relation 
with const an t coefficients 

a n = 4- c 2 a„-2 + ' 1 - + c*a„_ t + F(n\ 

then every solution is of the form {af + 4, w }, where {af ! } is a solution of the asso¬ 
ciated homogeneous recurrence relation 

«n - £ba n .-i 4- cid r .-2 + 1 ’ ■ 4 r ;ta„- 
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EXAMPLE II 


Proof: Since {af‘‘} is a particular solution of the nonhomogeneous recurrence relation, 
we know that 

= rfi, + C 2 $l 2 + ■■■ + c k af_ k + Pin). 

Now' suppose that {b,,) is a second solution of the nonhomogeneous recurrence relation, 
so that 

K = C[b n . i + c 2 b n . 2 + ■ ■ ■ + c k b n . k 4 F(n). 

Subtracting the first of these two equations from the second show's that 

b n ~ a'f' - c\(b n | - ) + c%(b n ~2 ~ + "' + c fJ>n-k ~ ^-0- 

It follows that {b„ - df } ) is a solution of the associated homogeneous linear recurrence, 
say. {a„ }. Consequently, b„ = af + a^' for all n. □ 

By Theorem 5, we see that the key to solving nonhomogeneous recurrence relations 
with constant coefficients is finding a particular solution. Then every' solution is a sum of 
this solution and a solution of the associated homogeneous recurrence relation. Although 
there is no general method for finding such a solution that works for every function Fi n), 
there are techniques that work for certain types of functions Fin), such as polynomials 
and powers of constants. This is illustrated in Examples 10 and 11. 


Find all solutions of the recurrence relation a* = 3a„- | + 2 n. What is the solution with 
di = 3? 

Solution: To solve this linear homogeneous recurrence relation with constant coef¬ 
ficients, we need to solve its associated linear homogeneous equation and to find a 
particular solution for the given nonhomogeneous equation. The associated linear ho¬ 
mogeneous equation is a„ = 3a„-|. Its solutions are a„ h> - a3 n , where or is a con¬ 
stant. 

We now find a particular solution. Since F{n) = In is a polynomial in n of degree 
one, one reasonable trial solution is a linear function in «, say, p„ = cn + d. where c 
and d are constants. To determine whether there are any solutions of this form, suppose 
that p n = cn 4 d is such a solution. Then the equation a„ = 3a,,-, + 2n becomes 
cn + d - 3(c(>t - 1} 4 d) + 2 n. Simplifying and combining like terms gives (2 + 
2c)n -+■ (2 d 3c) — 0, It follows that cn 4 d is a solution if and only if 2 -t- 2c — 0 and 
2d ~ 3r = 0, This shows that cn -r d is a solution if and only if c = -1 and d = -3/2. 
Consequently, a 1 /' 1 -■ -n - 3/2 is a particular solution. 

By Theorem 5 all solutions are of the form 

a„ - df + df' = n ~ l 4 a ■ 3" 
where a is a constant. 

To find the solution with ct] -- 3, lei n = 1 in the formula we obtained for the 
general solution. We find that 3^-1- 3/2 1 3a, which implies that a = 11/6. The 
solution we seek is a n -- - n 3/2 ^ (11/6)3*. ■ 


Find all solutions of the recurrence relation 
tin 5 «b- i — 6 a„... 2 + 7' 1 . 


PDF created with pdfFactory trial version www.pdffactorv.com 



m 


5 i Advanced Counting Techniques 


Solution: This is a linear iwnhomogctieous recurrence relation. The solutions of its 
associated homogeneous recurrence relation 

a„ -■ 5 a„ ] - 60„-2 

are a),'” = a\'3 n +ct2-2”, where a j and at are constants. Since /''(n) = 7", a reasonable 
trial solution is af' = C -7", where C is a constant. Substituting the terms of this 
sequence into the recurrence relation implies that C 7" - 5C - 7"~ l - 6 C - 7 ft " 2 4- 7". 
Factoring out 7 n_ \ this equation becomes 49(1' =• 35C - 6 C + 49, which implies that 
20C - 49, or that C - 49/20. Hence, 4f J - (49/20)7" is a particular solution. By 
Theorem 5, all solutions are of the form 

a„ - a i ■ 3" 4 - a 2 • 2" -t- (49/20)7". ■ 


In Examples 10 and 11, we made an educated guess that there are solutions of a 
particular form. In both cases we were able to find particular solutions. This was not 
an accident. Whenever F(n) is the product of a polynomial in n and the «th power of a 
constant, we know exactly what form a particular solution has, as stated in Theorem 6. 
We leave the proof of Theorem 6 as a challenging exercise for (he reader. 

THEOREM 6 Suppose that {a*} satisfies the linear Dontomogeoeous leajrreaas rotation 

o n = c } a„-i + i'2<h-2 + Cia*-k + Hn), 
where cj, C 2 ,...,c* are real numbers and ri • ; 

F(n) = (h ( n ! + + ■ • ■ 4- bin + bo)s n , 

where % fo. iy and s are real numbers. When s is net $ root of the character¬ 

istic equation of the associated linear homogeneous lecott^e relation, there is a 
particular solution of the form 

(pin 1 + p t ~ i n ,_1 + ■ • • + pj n + p< i )s’ 1 . 

When s is a root of this characteristic equation and its multiplicity is m, there is a 
particular solution of the form 

nHptd + p,^j n *" 1 + • ■ • + pj n 4- po)s*. 

Note that the factor ri" in the case when t is a root of multiplicity m of the charac¬ 
teristic equation of the associated linear homogeneous recurrence relation ensures that 
the proposed particular solution will not already be a solution of the associated linear 
homogeneous recurrence relation. We next provide an example to illustrate the form of 
a particular solution provided by Theorem 6 . 


What form does a particular solution ol the linear nonhoinogeneous recurrence relation 
a n - 6 tt n . [ - 9a„.. 2 + F(n) have when F(n) - 3". /■(«) = «3 B , F(n) = » 2 2", and 
F(n) = (n 2 + 1)3"? 

Solution: The associated linear homogeneous recurrence relation is a„ ^ 6a „ 1 - 
Wi„_ j. Its characteristic equation, r - 6 r 4- 9 - (r - 3 ) 2 = 0, has a single root, 3, of 
multiplicity two. To apply Theorem 6 , with Fin) of the form P(n)s n , where Pin) is a 
polynomial and s is a constant, we need to ask whether s is a root of this characteristic 
equation. 
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Since s = 3 is a root with multiplicity m = 2 but .r - 2 is not a root. Theorem 6 
tells us that a particular solution has the form V- if F(n) = 3", the form n 2 \pyn i 
/?o)3" if Fin) = ai n , the form {pjn 2 + p)tt + pu)2 n if F(n) - « 2 2 n , and the form 
+ p\n + p,)) 3” if F(n) - ( n 2 + 1)3". ■ 


Care must be taken when s = 1 when solving recurrence relations of the type cov¬ 
ered by Theorem 6 , In particular, to apply this theorem with F(n) = b t n t f b t .., n, , + 
■■■ + b]H + br u the parameter $ takes the value s = 1 (even though the term 1 " does not 
explicitly appear). By the theorem, the form of the solution then depends on whether J 
is a root of the characteristic equation of the associated linear homogeneous recurrence 
relation. This i.s illustrated in Example 13, which shows how Theorem 6 can be used to 
find a formula for the sum of the first n positive integers. 


EXAMPLE 13 Let a„ be the sum of the first n positive integers, so thal 

«„ = ]>> 

k-l 

Note that a R satisfies the linear nonhomogeneous recurrence relation 

a n = a„ | 4- tt. 

(To obtain a„. the sum of the first n positive integers, from the sum of the first 
n- 1 positive integers, we add n.) Note that the initial condition is a\ = 1. 

The associated linear homogeneous recurrence relation for a„ is 

a„ =«„_!. 

The solutions of this homogeneous recurrence relation are given by u^ 1 = 0(1)" - £'■ 
where c is a constant. To find all solutions of a„ = a n } + a. we need find only a 
single particular solution. By Theorem 6 , since F(n) = n = n ■ (1)" and s = lisa 
root of degree one of the characteristic equation of the associated linear homogeneous 
recurrence relations, there is a particular solution of the form n(p] nA p Q ) = pitr-r p a n . 

Inserting this into the recurrence relation gives p\?r +- p$n ~ pi(rt- l ) 2 + {n 
1) ■+ «■ Simplifying, we see that n(2p\ - 1 }+{po ~p i) = 0, which means that 2p ( - 1 - 
0 and po - p< = 0 , so pa - p\ = 1 / 2 . Hcncc, 

JP> = n ' n + ') 

u n ^ ' 2 ? 

is a particular solution. Hence, all solutions of the original recurrence relation «„ = 
a n -\ + n lire given by a n = 4- a { „ p) = c + n(n + l)/2. Since a, = 1 , we have 

I = a\ - c t 1 ■ 2/2 = c + i, so c - 0. It follows that a„ = n(n + 1 )/2, (This is the 
same formula found in Exercise 22 in Section 1.7.) ■ 


Exercises 


1. Determine which of the following are linear homoge¬ 
neous recurrence relations with constant coefficients. 
Also, find the degree of those that are. 


a) — 3a n -i + 4 a „_2 -I- 5d n .. 3 

b) a„ — 2na n . ] + a n -2 

c) a n = a „-1 + a„ 4 
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d) a, - o n -t t2 e) a„ = <t a ._, + a „_2 

n (i, a„ , 2 

g) a„ - a„- I + « 

2. Determine which of the following are linear homoge¬ 
neous recurrence relations with constant coefficients. 
Also, find the degree of those that are. 

a) o„ =• 3£!„-;> b) a n - 3 

c) a„ - al_ | d) a„ = a a -\ + 2a„_.i 

e) u„ = u„ yin f) fi„ = a„_[ + a„-j + n + 3 

g) a„ - + 5iv ^ + 9a„ 7 

3. Solve the following recurrence relations together with 
the initial conditions given. 

a) a„ = 2a„ i torn i i T op = 3 

b) a„ = a„ . I for a 1, hq — 2 

c) =■ 5« n _] » 6a„-j for ft s 2, df, = 1, a, =0 

dl A,, = 4ti„ ; - 4«„_i torn 2. an = 6. a] = 8 

e) ti„ - -4a,, i - for n £ 2, an = 0. a\ = 1 

fl d n - 4tj rl 2 for n £ 2. no = 0, «j = 4 

g) a„ = a„-il 4 forn e- 2, oq - !,ai - 0 

4. Solve the following recurrence relations together with 
the initial conditions given. 

a) a, - a„_i + 6a n _; for n £ 2, fl(, = 3, «, = 6 

h) - 7a„ ; - I0a„-i for n 2 2, tin = 2, m = 1 

c) a„ - 6a,, i - Sa .,-2 torn a 2, an — 4.a, - 10 

d) a„ - 2a„.] - a „-2 forn -- 2, no = 4,«( = 1 

e) a R = a., j forn ' 2,ao = 5, a\ = -1 

f) a„ = -6a n . i - 9a„ < for /i £ 2, an = 3, 
tfi - -3 

g) a„-i = + 5 a„ for n £ 0, = 2, a\ = K 

5. How many different messages can be transmitted in n 
microseconds using the tW'n signals described in Exer¬ 
cise 2.0 of Section 5.1? 

6. How many different messages can be transmitted in n 
microseconds using three different signals if one signal 
requires 1 microsecond for transmittal, the other two sig¬ 
nals require 2 microseconds each for transmittal, and a 
signal in a message is followed immediately by the next 
signal 11 

7. In how many ways can a 2 x n rectangular board be tiled 
using 1 x 2 and 2X2 pieces? 

8. A model for the number of lobsters caught per year is 
based on the assumption that the number of lobsters 
caught in a year is the average of the number caught in 
the two previous years. 

a) Find a recurrence relation for jL„), where is the 
number of lobsters caught in year n. under the as¬ 
sumption for this model. 

b) Find L„ if 100,00ft lobsters were caught in year I and 
300.000 were taught in year 2. 

4. A deposit of $100,000 is made in an investment fund at 
the beginning of a year. On the last day of each year two 
dividends arc awarded. The first dividend is 20% of the 
amount in the account during that year. The second divi¬ 
dend is 45','t of the amount in the account m the previous 
year. 


a} Find a recurrence relation for {?„}, where P„ is the 
amount in the account at the end of n years if no 
money is ever withdrawn. 

b) How much is in the account after n years if no 
money has been withdrawn? 

*11). Prove Theorem 2. 

11 . The Lucas numbers satisfy the recurrence relation 

L, = I„-t + U-i. 

and the initial conditions U } - 2 and L\ = I, 

a) Show that L„ = /„_] + f r .n for n = 2,3,..., 
where /„ is the nth Fibonacci number, 

b) Find an explicit formula for the Lucas numbers. 

12 . Find the solution to a„ = 2a„. i + a„ 2 ~2a„ . < for n = 

3, 4,5.with flo = 3, «i = 6, and a? = 0. 

13 . Find the solution to o„ - 7 a „~2 + 6(i„- 1 with «r, - 9, 
fli = 10, and ai = 32. 

14 . Find the solution to a„ = 5a 4 -2 ~ 4a„_4 with u n - 3, 
fl] = 2, a-} = 6, and Q\ = 8. 

15 . Find the solution to a n = 2a„- - 6<; rt _ : with 

uq = 7, ai = -4, and u; = 8, 

* 16 . Prove Theorem 3. 

17 . Prove the following identity relating the Fibonacci 
numbers and the binomial coefficients: 

f„ +i = C(n , 0) + Cffl - 1,1) + ■ - ■ + C(n - k, k) 

w here n is a positive integer and k = [n/2j. [Hint: Let 
u„ - C{n, 0) + C(n - 1, 1) + ■ ■■ + C(rt - k, k). Show 
that the sequence {akt satisfies the same recurrence re¬ 
lation and initial conditions satisfied by the sequence 
of Fibonacci numbers, ] 

18 . Solve the recurrence relation <j„ = 6ri„. 1 - 12«a n -2 + 

with no = -5. a| = 4,andi7; = 88 

19 . Solve the recurrence relation a„ ~ -3 a„ 1 - 3o'„ . - 

a „-2 with rig = 5, = -9, andtij : = 15. 

20. Find the general form of the solutions of the recurrence 

relation a., = 8u„ - 16ri f ,-,i. 

21 . What is the general form of the solutions of a lin¬ 
ear homogeneous recurrence relation if its charac¬ 
teristic equation has roots, 1, 1, 1, 1, -2, -2. -2,3, 
3, -4? 

22. What is the general form of the solutions of a linear 
homogeneous recurrence relation if in, characteristic 
equation has the roots -1, -1, -1,2, 2.5,5.7? 

23. Consider the nonhomogeneous linear recurrence rela¬ 
tion a„ = 3(!„ 1 + 2", 

aj Show 1 that a„ ~ -2' w * 1 is a solution of this recur¬ 
rence relation, 

b) Use Theorem 5 to find all solutions of this recur¬ 
rence relation. 

c) Find the solution with « 0 = 1. 

24. Consider the nonhomogeneous linear recurrence rela¬ 
tion n„ = 2a„-i + 2". 

a) Show' that a„ ~ n 2” is a solution of this recurrence 
relation. 


PDF created with pdfFactory trial version www.pdffactorv.com 



5.2 Exercises 33 i 


b) Use Theorem 5 to find all solutions of this recur¬ 
rence relation. 

cf fund flic solution with tm = 2 . 

25. a) Determine values of the constants .3 and B so that 

a„ = An F B is a solution of the recurrence relation 
a„ — 2a „-1 f n t 5. 

b) Use Theorem 5 to find all the solutions of this re¬ 
currence relation. 

cl Find the solution of this recurrence relation with 
flo = 4. 

26. What is the general form of the particular solution of 
the linear nonhomogcncous recurrence relation a„ ~ 
6 a n i - I 2 b„ 2 ~ 8 (v„ 3 + Bin) if 

a) FI u) = rr? 

b) Fin} = 2"? 

c) fin) - n 2 "? 

d) Fin) = (-2T? 

C) Fin) = rf2 r ' ,> 

f) Fin) = it i 

g) F(n) = 3? 

27. What is the general farm of the particular solution of 
the linear non homogeneous recurrence relation a„ 

8 a„ : - 1 fwi:, 4 + F(n) if 
a.) Fin) = rt ! ? 

b) Fin) = (-23"? 

c) Fin) - n2"7 

d) Fin) — /j 2 4 b 7 

et /fu; •• in 2 )( 2 r? 
f) F(n) =■= n 4 2 n 7 
fi) T(n) -= 2 11 

28. a) Find all solutions of the recurrence relation a n = 

2a, t i Fin 1 . 

bj Find the solution of the recurrence relation in part 
(a) with initial condition a\ = 4. 

29. ai Find all solutions of the recurrence relation a x - 

2« n ., +3”. 

b) Find the solution of the recurrence relation in part 
(aj with initial condition «i - 5. 

30. a) Find all solutions of the recurrence relation a„ - 

| - 6 a,, i + 42 • 4' 1 . 

bl Find the solution of this recurrence relation with 
a | — 56 and a ; -- 278. 

31. Find all solutions of the recurrence relation a„ ■ 
5ij„ | - ba„ ; t? f 3 n. (Him: Look for a particu¬ 
lar solution of the form qnY' -t pjtt + />:, where ah Pb 
and />2 are constants.) 

32. Find the solution of the recurrence relation a„ ^ 
2a„. i +3-2T 

33. Find all solutions of the recurrence relation a„ - 
4a,, i - 4(i ft j ) in i 1)2". 

34. Find all solutions of the recurrence relation a„ ~ 
7<v, - 16(7.„-2 + 12u„-!-«4" with®. - -2,0] - 0. 
and a 2 = 5. 

35. Find the solution of the recurrence relation n, = 
4«„.. | ■ Ju,,.-; + 2" + n + 3 with tin - 1 aodui - 4 


36. Let a n be the sum of the first n perfect squares, that 
is, a„ = VJ v] k 2 . Show that the sequence {a„} sat¬ 
isfies the linear nonhomoge neons recurrence relation 
a„ - £j„..| + n 1 and the initial condition a ] = I, Use 
Theorem 6 to determine a formula kiia„ by solving this 
recurrence relation. 

37. Let a„ be the sum of the first n triangular numbers, that 
is, a» = i k , where r* = k(k + l)/2. Show that 
|a„} satisfies the linear nonhomogeneous recurrence re¬ 
lation a„ = a„ ~ 1 + n(n + l )/2 and the initial condition 
;j| = 1. Lise Theorem 6 to determine a formula for a„ 
by solving tills, recurrence relation. 

38. a) Find the characteristic roots of the linear homo¬ 

geneous recurrence relation a„ = 2a„ , - 2 a„ 
(Note; These are complex numbers.) 
b) Find the solution of the recurrence relation in part 
(a) with bo = I anddi = 2 . 

*39. a) Find the characteristic roots of the linear homoge¬ 
neous recurrence relation a„ = a n _ 4 . [Note: These 
include complex numbers ; 

b) Find Lhe solution of the recurrence relation in part 
(a) with oo = l.£i[ = 0 , fl 2 = -l.andBj = 1 . 

*40. Solve the simultaneous recurrence relations 


a„ ~ ia n - + 26 a -1 
h„ = a n ; + 2b„ , 
with on 1 and - 2 . 

*41. a) Use the formula found in Example 4 for f„, the nth 
Fibonacci number, to show that j| is the integer 
closest to 


J5 


(¥) 


h) Determine for which a f„ is greater than 


U5 

and for which n f„ is less than 


1 + ,/5 
2 


42, Show that if u„ - - s and a■ - 

where r and t are constants, then a„ ~ sf.,-\ i ff H for 
all positive integers n, 

43. Express the solution of the linear nonhomogenous re¬ 
currence relation a a = a„. [+a, jt-iforn x 2 where 
bo = Oandtij = 1 in terms of the Fibonacci numbers. 
(Him Let b„ — a„ I and apply Exercise 42 to the 
sequence b„.) 

*44, (Linear algebra required) Let A.,, be the n X n matrix 
with 2 s on its main diagonal. Is in all positions next 
to £ diagonal element, and Os everywhere else. Find a 
recurrence relation for d„, the determinant of A„. Solve 
this recurrence relation to find a fttmiula for d„ 
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45. Suppose that cadi pair of a genetically engineered 
species of rabbits left on an island produces two new 
pairs of rabbits at the age of E month and six new pairs 
of rabbits at the age of 2 months and every month after¬ 
ward. None of the rabbits ever die or leave the island 

a) f ind a recurrence relation for the number of pairs 
of rabbits on the island n months after one newborn 
pair is left on the island. 

b) By solving the recurrence relation in (a) determine 
the number of pairs of rabbits on the island n months 
after one pair is left an the island. 

4b. Suppose that there are two goats on an island ini¬ 
tially. The number of goats on the island doubles ev¬ 
ery year by natural reproduction, and some goats are 
either added or removed each year. 

a) Construct a recurrence relation for ihe number of 
goats on the island at the start of the nth year, as¬ 
suming that during each year an extra 100 goats are 
put on the island, 

b) Solve the recurrence relation from part (a) to find 
the number of goats on the island at the start of the 
nth year. 

ci Construct a recurrence relation for the number of 
goats nn the island at the start of the nth year, as¬ 
suming that n goats are removed during the nth 'ear 
for each n i 1 

d J Soi ve the recurrence relation in part (c) for the num¬ 
ber of goats on the island at the start of the nth year. 

47. A new employee at an exciting new software company 
starts with a salary of 550,000 and is promised that at 
the end of each year her salary will be double her salary 
the previous year, with an extra increment of $ 10,000 
for each year she has been with the company, 

») Construct a recurrence relation for her salary for her 
nth year of employment, 

b l Solve this recurrence relation to find her salary for 
her nth year of employment. 


Some linear recurrence relations do not have constant coef¬ 
ficients which can be systematically solved. This is the case 
for recurrence relations of the form f(n)a„ - g(n)a„ _ ] -t 
k(n). Exercises 48-50 illustrate this, 

*48, at Show that the recurrence relation 
f(n)a, t = g(n)a a . i + h(n ), 

for nil, and with Op - C, can be reduced to a 
recurrence relation of the form 

b„ - b„ i + Qin)h{n), 

where b„ - gin ~ l)£J(n 4 1 )d„, with 

£?{») = l/fl t/{2)••■/(«- I>)/(£(f)g(2j ■ ■ - g(n)l 

b) Use pari (a) in solve the original recurrence relation 
to obtain 

a = C l2 h ifrtj_<2KOW) 

+ 1) ’ 

*49. Use Exercise 4b to solve the recurrence relation in + 
l>„ = [n f 3)a„ [ + n, for n o I, with oq = I. 

50. It can be shown that the average number of com¬ 
parisons made by the quick sort algorithm (described 
in the exercise set of Section 8.4), when sorting n 
elements in random order, satisfies die recurrence 
relation 

2 Itrf 

Ctj = fi t i h — ^ 

” irtj 

torn = 1,2.with initial condition C(i - 0. 

a) Show that {C„} also satisfies the recurrence relation 

nC„ = (rt +- ljr„_L + 2n for n = 1,2. 

b) Use Exercise 48 to solve the recurrence relation in 
part (a) to find an explicit formula for C„ 

**5I. Prove Theorem 4. 

**52. Prove Theorem 6. 


5.3 __ 

D i v i de- an d-Co nq ue r Relations 

INTRODUCTION 


Many recursive algorithms take a problem with given input and divide it into one or 
more smaller problems. This reduction is successively applied until the solutions of the 
smaller problems can be found quickly. For instance, we perform a binary search by 
reducing the search for an element in a list to the search for this dement in a list half as 
long. We successively apply this reduction until one element is left. Another example of 
this type of recursive algorithm is a procedure for multiplying integers that reduces the 
problem of the multiplication of two integers to three multiplications of pairs of integers 
with hall as many bits. This reduction is successively applied until integers with one 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


bit are obtained. These procedures are called divide-and-conquer algorithms. In this 
section the recurrence relations that arise in the analysis of the complexity of these 
algorithms will be studied. 


DIVIDE-AND-CONQUER RELATIONS 


.Suppose that an algorithm splits a problem of size n into a sttbprobkms, where each 
subproblem is of size nib (for simplicity, suppose that b divides n; in reality, the smaller 
problems are often of size equal to the nearest integer either less than or equal to, or 
greater than or equal to, nib). Also, suppose that a total of g(n) extra operations are 
required when this split of a problem of size n into smaller problems is made. Then, if 
/(ft) represents the number of operations required to solve the problem, it follows that 
/ satisfies the recurrence relation 

/(n) = af (nib) + g(n). 

This is called a divide-and-conquer recurrence relation. 


We introduced a binary search algorithm in Section 2,1. This binary search algorithm 
reduces the search for an element in a search sequence of size n to the binary search for 
this dement in a search sequence of size nil, when n is even. (Hence, the problem of 
size n has been reduced to one problem of size nil.) Two comparisons are needed to 
implement this reduction (one to determine which half of the list to use and the other 
to determine whether any terms of the list remain). Hence, if fin) is the number of 
comparisons required to search for an element in a search sequence of size n. then 
fin) - /(nil) + 2 when n is even. ■ 


Consider the following algorithm for locating the minimum and maximum dements 
ot a sequence S|, a 2 ..... a n . if n - 1, then aj is the maximum and the minimum. If 
n > 1, split the sequence into two sequences, either where both have the same num¬ 
ber of elements or where one of the sets has one element more than the other. The 
problem is reduced to finding the maximum and minimum of each of the two smaller 
sequences. The solution to the original problem results from the comparison of the sep¬ 
arate maxima and minima of the two smaller sets to obtain the overall maximum and 
minimum. 

Let fin) be the total number of comparisons needed to find the minimum and max¬ 
imum elements of the set with n elements, We have shewn that a problem of size n can 
be reduced into two problems of size nil, when n is even, using two comparisons, one 
to compare the minima of the two sets and the other to compare the maxima of the two 
sets. This gives the recurrence relation fin) = If {nil) + 2 when n is even. ■ 


Surprisingly, there are more efficient algorithms than the conventional algorithm (de¬ 
scribed in Section 2.4) for multiplying integers,. One of these algorithms, which uses 
a divide-and-conquer technique, will be described here. This fast multiplication algo¬ 
rithm proceeds by splitting each of two 2n-bit integers into two blocks each with n 
bits. Then, the original multiplication is reduced from the multiplication of two 2mbit 
integers to three multiplications of ri-bit integers, plus shifts and additions. 
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EXAMPLE 4 


Suppose that a and b are integers with binary expansions of length In (add initial 
bits of zero in these expansions if necessary to make them the same length). Let 

a = {aj„-ifl 2 n -2 ‘■■a^aoh and b = lb 2n ^ 1 * 0 ) 2 . 


Let 


u — 2 n A] + Ao, b = 2 n B\ r Bg, 
where 

til = . "-i 3 ffl+l«n) 2 , Af, = («„-! ■■■ 01^)2, 

^1 = {bi n . 1 ■■ ■ b„+\b n )2, So ■- (h n -j - b[b ci) 2 . 

The algorithm for fast multiplication of integers is based on the identity 
ab ~ O' 171 + 2 n )A!fii + 2 n (A, - A 0 )(£ 0 - B t ) + (2 n + 1)A 0 S 0 . 

The important fact about this identity is that it shows that the multiplication of two 
2 / 2 -bit integers can be carried out using three multiplications of «-bit integers, together 
with additions, subtractions, and shifts. This show's that if / (n) is the total number of 
bit operations needed to multiply two mbit integers, then 

I (In) - 3/(n) + Cn. 

The reasoni ng behind this equation is as follows. The three multiplications of n-bit inte¬ 
gers are carried out using 3/(rt)-bit operations. Each of the additions, subtractions, and 
shifts uses a constant multiple of n-bit operations, and Cn represents the total number 
of bit operations used by these operations. ■ 


There are algorithms that multiply two n '< n matrices, when n is even, using seven 
multiplications each of two (n/2) X (n/2) matrices and 15 additions of (nil) x (n/2) 
matrices, Hence, if f(n) is the number of operations (multiplications and additions) 
used, it follows that 

f(n) = If (nil) + I5ir/4 

when n is even. ■ 


As Examples 1-4 show', recurrence relations of the form f(n) = af(nfh) + gin) arise 
in many different situations. It is possible to derive estimates of the size of functions 
that satisfy such recurrence relations. Suppose that/ satisfies this recurrence relation 
whenever n is divisible by b. Let n = b k , where k is a positive integer. Then 

/(«> = af(nfb) + g(n) 

= a 2 fin lb 1 ) + ag(ri lb) + gin) 

= a 2 f(n/b y ) + a 2 g(nlb 2 ) + ag(n/b) 4- g(n) 


Jt-I 

■ a k f{nib k ) + y]a J g(nl b J ). 
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Since n/h* = 1, it follows that 

/{«) = a k f{\) + Ya J g(n/b J ). 

i =o 

We can use this equation for fin) to estimate the size of functions that satisfy divide- 
and-eonquer relations. 


THEOREM 1 


Let/ bean inatastag fmtfm feat aisfa© fee 
f(n) = af(ntb) + c > : "f'-f f 


whenever ji is divisible by where a 2 l, £i&aa nttgcrgtqatet' thaaK aa<U isa 

positive real number. Then 


/( „). i 

nl 100og«) 


if a > 1, 
if a = J. 


Proof: First let n - b k . From the expression for fin) obtained in the discussion pre¬ 
ceding the theorem, with g(n) = c, we have 

Jt-l A-l 

fin) = a k f{ 1) + X (,Jc = a*/(0 + cYd. 

>=o /=c 

First consider the case when a -- I. Then 

/(«) = /(1) + ck. 

Since n = b l '\ we have k - log fc n. Hence 
/(«) = /(!) + c log A n. 

When n is not a power of b, we have b k < n < h k *\ for a positive integer k. Since/ is 
increasing, it follows that /(«) f(b k+l ) = /{1) + c(k + 1} = {/(l) + c) + ck £ 
(/(1} + c) + c log t n. Therefore, in both cases, fin) is Oflog n) when a = 1. 

Now suppose that a > I. First assume that n - b k where it is a positive integer. 
From the formula for the sum of tenns of a geometric progression (Example 6 of Section 
3.2), it follows that 

fin) ^/(D + cfo*- \)({a- 1) 

= a k [f(l) + cf(a - 1)1 - d(a - 1) 

= Ci n lo$i “ + C 2r 

since a k - a ]os »' 1 - n lf,gt3 (see Exercise 4 in Appendix 1), where C\ = [/(l }+d 
(a- 1}] andC 2 = -d{a~ 1). 

Now suppose that n is not a power of />. Then b k < n < h k ' ] where k is a nonneg¬ 
ative integer. Since/ is increasing, 

fin) - /(***') =- C ia M +C 2 

« :r : okr”' " + C 2 

3 (C 2 a)n ]0 ^ f c 2 , 

since k == log/, n < k + 1, 

Hence, we have fin) is O(n ,0& “), □ 
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EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


THEOKEM 2 


Remark: This proof gives an explicit formula for /(«) where n = b k . 
The following examples illustrate how Theorem I is used. 


Let /(«} - 5/(it / 2) + 3 and /(1) - 7. Find /(2*) where k is a positive integer. Also, 
estimate fin) iff is an increasing function. 

Solution: From the proof of Theorem 1, with a = 5, b = 2, and c - 3, we see that if 
n - 2/ then 

f(n) = Am + a (a - Q] + [-C/fa - i)| 

- 5*[7 + (3/4)] - 3/4 

- J5*'(31/4) - 3/4. 

Also, if /(n) is increasing. Theorem 1 show's that f(n) is = 0(n k,ii ). ® 


We can use Theorem 1 to estimate the computational complexity of the binary search al¬ 
gorithm and the algorithm given in Example 2 for locating the minimum and maximum 
of a sequence. 


Estimate the number of comparisons used by a binary search. 

Solution: In Example 1 it was shown that fin) - f(n/2) + 2 when « is even, where 
f is the number of comparisons required to perform a binary search oti a sequence of 
size ft. Hence, from Theorem 1, it follows that f(n) is 0(log n). ■ 


Estimate the number of comparisons used to locate the maximum and minimum ele¬ 
ments in a sequence using the algorithm given m Example 2. 

Solution: In Example 2 we showed that /(«} - 2/(«/2) + 2, when n is even, where 
/ is the number of comparisons needed by this algorithm. Hence, from Theorem 1, it 
follows that f(n) is 0(n k,g2 ) = Oin). ■ 


We will now state a more general, and more complicated, theorem that is useful in 
analyzing the complexity of divide-and-eonquer algorithms. 

Let/ be an increasing function that satisfies the recurrence relation 

/(») = afinib) + cr^ - ■ 

whenever n - b k , where k is a positive integer, a s I, b is an integer greater than 
1, and c and d are positive real numbers. Then 

[OO^) 

/(") = \ Ofn^logn) if a-I/, 
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The proof of Theorem 2 is left for the reader as Exercises 17-21 at the end of this 
section. 


EXAMPLE 8 Estimate the number of bit operations needed to multiply two n-bit integers using the 

fast multiplication algorithm. 

Solution: Example 3 shows that /(«) = 3/(n/2) + Cn, when n is even, where fin) 
is the number of bit operations required to multiply two n-bit integers using the fast 
multiplication algorithm. Hence, from Theorem 2, it follows that fin) is Oin ^\ Note 
that log 3 - 1.6. Since the conventional algorithm for multiplication uses (An-) bit 
operations, the fast multiplication algorithm is a substantial improvement over the con¬ 
ventional algorithm in terms of time complexity for sufficiently large integers. ■ 

EXAMPLE 9 Estimate the number of multiplications and additions required to multiply two n X n 

matrices using the matrix multiplication algorithm referred to in Example 4. 

Solution: Let fin) denote the number of additions and multiplications used by the 
algorithm mentioned in Example 4 to multiply two n X. n matrices. We have f(n) --- 
If in 12) + 15« 2 /4, when n is even. Hence, from Theorem 2, it follows that /(n| is 
0{u lt,f '). Note that log 7 ~ 2,6. Since the conventional algorithm tor multiplying two 
nXn matrices uses (An 1 ) additions and multiplications, it follows that for sufficiently 
large integers n, this algorithm is substantially more efficient in time complexity than 
the conventional algorithm. ■ 


Exercises 


1 . How many comparisons are needed for a binary search 
in a set of 64 elements ' 

2. How many comparisons are needed to locate the max¬ 
imum and minimum elements in a sequence with 128 
elements using the algorithm in Example 27 

3. Multiply (III 0); and (1010); using the fust multiplica¬ 
tion algorithm. 

4. Express the fast multiplication algorithm in pseu¬ 
docode. 

5. Determine a value for the constant Tin Example 3 and 
use it to estimate the number of bit operations needed 
to multiply two 64-bit integers using the fast multipli¬ 
cation algorithm. 

6 . How many operations are needed to multiply two 32 x 
32 matrices, using the algorithm referred to in Exam¬ 
ple 41 

7. Suppose that fin) - fin! 3.) + 1 when n is divisible 
by 3, and /<I) = I. Find 

a) fif'i. 
h) fi2U 
e) /(729). 

8 . Suppose that fin) - 2f(ri/2) + 3 when n is evert, and 
/(l) - 5. Find 


a) /(Ti¬ 
bi f(t). 

c) f{ 64). 

d) f(102.4). 

9, Suppose that fin) = fin/5) + 3 n 1 when n is divisible 
by 3, and /flj = 4. Find 

a) f(5). 

b) /(125). 

O /t3l25). 

10. Find /(n) when n -- 2/ where/ satisfies the recur¬ 
rence relation fin) - f(nl2) -r 1 with /(I) = 1 . 

11. Estimate the size of/in Exercise 10 if/is an increasing 
function, 

12. Find /(n) when n - 3/ where / satisfies the recur¬ 
rence relation f[n) - 2/(h/ 3) r 4 with /(I) = 1. 

13. Estimate the size of/in Exercise 12 if/is an increasing 
function. 

14. Suppose that there are u = 2 ' teams in an elimina¬ 
tion tournament, where there are nil games in the first 
round, with the nil - 2 *' ! winners playing in the sec¬ 
ond round, and so on. Develop a recurrence relation for 
the number of rounds in the tournament. 
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15. Hew many rounds arc in the elimination tournament 
described in Exercise 14 when there are 32 teams? 

16. Solve the recurrence relation for the number of rounds 
in the tournament described in Exercise 14. 

in Exercises 17 21. assume that/ is an increasing function 
satisfying the recurrence relation fin) = aflnlb l + af, 
where a 1. b is an integer greater than 1, and c and d 
are positive real numbers. These exercises supply a proof 
of Theorem? 

*17. Show that if a >f and n is a power of b, then f(n) =■ 
/(IV t- eri' 1 logj, tt, 

18. Use Exercise 17 to show that if a ■= ft 1 '*, then f(n) is 
DV log it). 


'lb. Show' that if a t- If and n ts a power of b, then f(n) - 
C i >f + Ci n k ' £ t | *, where C ( = If c / (V - a) and C: = 
/(l) + tfc.Ha - lf). 

20. Use Exercise 19 to show that: if a < If, then /(h) is 
0(n d ), 

21. Use Exercise 19 to show that if a > b d , then /(w) is 
0(n l0Sha )- 

22. hind fin) when n - 4*, where / satisfies the recur¬ 
rence relation f(n) = 5/(n/4) + 6n, with /(l) = i. 

23. Estimate the size of/in Exercise 22 if f is an increasing 
function. 

24. Find f[n) when n = 2/ where / satisfies the recur¬ 
rence relation f(n) = 8/(n/2) + n 2 with /(1) - 1. 

25- Estimate the size off in Exercise 24 iff is an increasing 
function. 


5.4 _ 

Generating Functions 

INTRODUCTION 


Generating functions are used to represent sequences efficiently by coding the terms 
of a sequence as coefficients of powers of a variable x in a formal power series. Gen¬ 
erating functions can be used to solve many types of counting problems, such as the 
number of ways to select or distribute objects of different kinds, subject to a variety of 
constraints, and the number of ways to make change for a dollar using coins of differ¬ 
ent denominations. Generating functions can be used to solve recurrence relations by 
translating a recurrence relation for the terms of a sequence into an equation involv¬ 
ing a generating function. This equation then can be solved to find a closed form for 
the generating function. From this closed form, the coefficients of the power series for 
the generating function can be found, solving the original recurrence relation. Generat¬ 
ing functions can also be used to prove combinatorial identities bv taking advantage of 
relatively simple relationships between functions that can be translated into identities 
in volving the terms of sequences. Generating functions are a helpful tool for studying 
many properties of sequences beside those described in this section, such as their use 
for establishing asymptotic formulae for the terms of a sequence. 

We begin with the definition of the generating function for a sequence. 

DEFINITION 1. The generating function for the sequence oq, a\,. ,. , a*,... of 
real numbers is the infinite series 

{?(.*) = flu + ajx + * ■ ■ + a k x k + • ■ - = a k x k . 

k = Q 


Remark: The generating function for {a t } given in Definition 1 is sometimes called the 
ordinary generating function of {a*} to distinguish it from other types of generating 
functions for this sequence. 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


The generating functions for the sequences {a*} with a* - 3,^ -■ k + I, and a k = 2* 
are 2 ’i = t)3.c* r y t =0 (& + l)x , and V^ 0 2*rr,respectively, ■ 


We can define generating functions for finite sequences of real numbers by ex¬ 
tending a finite sequence oo,<3i ,. ..,a a into an infinite sequence by .setting a n+ \ = 
0, Qn +2 — 0, and so on. The generating function G{.x) of this infinite sequence {a„} is a 
polynomial of degree n since no terms of the form a } x J with j > n occur, that is, 

G(x) = flo +■ Q\x T ■ ■ ■ + a n .t rt . 


What is the generating function for the sequence 1,1,1, 1,1, 1? 

Solution: The generating function of L 1,1, L 1,1 is 

1 + jc + x 2 + x 3 + .r 4 + .J-. 

By Example 6 of Section 3.2 we have 

(x 6 - \')!(x - 1) = 1 + x + x 2 +- x 3 + x 4 + x 5 . 

Consequently, GO) = U e - l )/(x - 1) is the generating function of the sequence 1, 1, 

1 , 1 , 1 , 1 . ■ 


Let m be a positive integer. Let a k = C(m, k), for k = 0, 1 , 2 , ..., m. What is the gen¬ 
erating function for the sequence ag, a\ .a m ? 

Solution: The generating function for this sequence is 

G{x) = Cim, 0} + C(m, l)x + C{ra, 2)r + - ■ - + C(tn, m)x m . 

The binomial theorem shows that G(x) = (1 + x) m , ■ 

USEFUL FACTS ABOUT POWER SERIES 

When generating functions are used to solve counting problems, they are usually con¬ 
sidered to be formal power series. Questions about the convergence of these series 
are ignored. However, to apply some results from calculus, it is sometimes important 
to consider for which x the power series converges. We will not be concerned with 
questions of convergence in our discussions. Readers familiar with calculus can con¬ 
sult textbooks on this subject for details about the convergence of the series we consider 
here. 

We will state now 1 some important facts about infinite series used when working 
with generating functions. A discussion of these and related results can be found in 
calculus texts. 


The function /(x) = 1/(1 — x) is the generating function of the sequence LI, 1, 1,.,,, 
since 

1/(1 ~ jc ) - 1 + x + x* + -- 

for lx| < 1. ■ 
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EXAMPLE 5 


THEOREM 1 


EXAMPLE 6 


EXAMPLE 7 


The function f(x) ~ 1/(1 -ax) is the generating function of the sequence 1, a, a 2 , a'.. . 
since 

1/(1 - ax) = 1 +- ax + a 2 * 2 + ■ ■ ■ 

when |u,t| < 1, or equivalently, for |x| < l/|a| for a ¥= 0. ■ 

We also will need some results on how to add and bow r to multiply two generating 
functions. Proofs of these results can be found in calculus texts. 

Let f(x) ~ Xr=o<hW*andg(.j;) = Then 

fix) + six) ~ V(a k + b k )x l and f(x)g(x) = V[ V a A-v] x k . 

i= 0 M\j = 0 / 

We will illustrate how Theorem 1 can be used with the following example. 

Let fix) = 1/(1 - ,v)w Use Example 4 to find the coefficients a,-,. a ],«?.... in the ex¬ 
pansion f[x) = XL »«***■ 

Solution: From Example 4 we see that 

1/(1 - X ) = 1 + X + X 2 + JT + ' ■' . 

Hence, from Theorem 1, we have 

i/d -x? = x( X 1 
*-o\/=o 


= Y(k + \)x k , 
k- n 


Remark: This result also can be derived from Example 4 by differentiation. Taking 
derivatives is a useful technique for producing new' identities from existing identities 
for generating functions. 

To use generating functions to solve many important counting problems, we will 
need to apply the binomial theorem for exponents that are not positive integers. Be¬ 
fore we state an extended version of the binomial theorem, we need to define extended 
binomial coefficients. 


DEFINITION 2, Let u be a real number and k a nonnegative integer. Then the 
extended binomial coefficient (?) is defined by 

M = { «(« - 1) * • ■ (w - k + n/Jt! if k > 0, 

\kj [l if Jt = 0. 


Find the values of the extended binomial coefficients (j 2 ) and { il2 ). 
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EXAMPLE N 


THEOREM 2 


Solution: Taking u - -2 and k - 3 in Definition 2 gives us 

, Z\ _ •: 2,; 4) _ _ 

' 3 ! " 3' 

Similarly, taking u - 1/2 and k — 3 gives us 
/U2\ _ 0/2K1/2 '^IK_J/2_ _2) 

U/ = " ' ' 3 ! 

- (1/2)1-l/2)(-3/2)/6 
= 1/16. 


Tlie following example provides a useful formula for extended binomial coeffi¬ 
cients when the top parameter is a negative integer. It will be useful in our subsequent 
discussions. 


When the top parameter is a negative integer, the extended binomial coefficient can 
be expressed in terms of an ordinary binomial coefficient. To see that this is the case, 
note that 

|'-rr\ {-w)(— n ~ 1) ‘ (-11 - r -t- 1) 

l , j ~ r! 

( -l) r n(n + 1) ■ ■- (« i - r - ■ 1) 

_ 

l 1 i r (/i • ' I :(u ■ r 2i i; 

..... ... ...... ^ 

(-■ l) r (« + r - 1)1 

>■•:/! -Ijj 

, , in + r - 1 \ 

“ Ml ( ■ ) 

- I, - iY Ct i; i r - 1, r), % 


We now state the extended binomial theorem. 

THE EXTENDED BINOMIAL THEOREM Let .v be a real number with |xj < i 
and let u be a real number. Then 

<1+Jr) “ 

Theorem 2 can be proved using the theory of the Maclaurin series. Wc leave its 
proof to the reader with a familiarity with this part of calculus. 

Remark: When n is a positive integer, the extended binomial theorem reduces to lire 
binomial theorem presented in Section 4.3, since in that case ft) - 0 if k > u. 
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EXAMPLE 9 


EXAMPLE 10 


The following example illustrates the use of Theorem 2 when the exponent is a 
negative integer. 


Find the generating functions for (1 + .r’r" and (1 -- i) * where n is a positive integer, 
using the extended binomial theorem. 

Solution: By the extended binomial theorem, it follows that 
<1 + ,r '-§;(?)**• 

Using Example 8, which provides a simple formula for we obtain 

(! + x) fl - V(-l) fc C(/i t k I k)x k . 

*=»o 

Replacing x by — jc. we find that 

(1 -x)*' 1 - y C(n + k - I. k)x\ 

■ 


Table 1 presents a useful summary of some generating functions that arise fre¬ 
quently. 


COUNTING PROBLEMS AND GENERATING FUNCTIONS 

Generating functions can be used to solve a wide variety of counting problems. In par¬ 
ticular, they can be used to count the number of combinations of various types. In Chap¬ 
ter 4 we developed techniques to count the /--combinations from a set with n elements 
when repetition is allowed and additional constraints may exist. Such problems are 
equivalent to counting the solutions to equations of the form 

+ &2 + ■ 1 ' +■ = C t 

where C is a constant and each e, is a nonnegative integer that may be subject to a 
specified constraint. Generating functions can also be used to solve counting problems 
of this type, as the following examples show. 


Find the number of solutions of 

t\ + €2 + Cj - 17, 

where eu «i, and e\ are nonnegative integers with 2 £ ti s 5,3 £ e 3 < 6, and 4 s 

7 . 

Solution: The number of solutions with the indicated constraints is the coefficient of 
x {1 in the expansion of 

(a 2 + + X 4 + A’ 5 )fr + A' 4 4- X s + a 6 !(.C + X 5 4 ,v' J 4- Jt 7 ). 
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TABLE 1 ffsefuJ Generating Functions. 

G[x) 


' 1 4 .U" - C(n, k) x* 

k=(l 

= 1 + C{n, IJ.v + C(n, 2)x' t + -f t" 

C(n, k) 

( 1 + ax}* - V C(n. k)a\x k 
t: 0 

- 1 4 C(«, 1)<ja 4 C(n, 2)a 3 x 2 + - ■ • 4- a n x" 

C(n, k)a r 

(1 + *T = V C(n, k)x A 

k- 0 

= ! 4 C(n, l)x r + C(n< 2U* F + * ** + -r™ 

C(n. k!r) if r\k\ 0 otherwise 

1 . --f x k ! • t • t ' * • • tr 

1 - *=s 

1 if k £ n\ 0 otherwise 

= V x k = I4r4a 2 4- 

1 “ - l PS 

I 

t * i- 

7 -- 'V & k x k = I + ax 4 £T.r 4- ■ ■ ■ la* 

1 " njf U 1 

--'S x tIl = 1 + X T 4 x* r 4 r ‘ ■ 

1 Jcr rs 

] if rjk; 0 otherwise 

- -vs - 'S (k 4 i )X k = 1 + 2x + 3 X~ -h " 1 

— 1 

k + 1 

■■ ■-■ - - V an + k-lk)x k \ 

■■ 1 4 C(n, l)x+C{n+ 1, 2)-r + ■■■ 

C(n 4 k- U) = C(/i4 it - l,n~ 1) 

- - 1 - - ' V C{n 4 k - 1, k ){-1 a* i 

■ + X * t »o | 

f’(n. lie - f '(n • 1. 2s .t 

C:n • 2.3|.t : • ••• 

aiyCU + it- I, it) = {~l)*C(n + Jfc- l.n - 1) 

i * 

-= > C{n+k- 1 

(i-r«r — ! 

■' l + f ’(n, 1 )a.r 4 Clri 4 [, 2')ci ’r + ■ ■ - 

C(n 4 k -■ I, k)a k = C(n 4- k - 1, n - 1 )a* 

. v' -c* .r ,r J 1 

1/E! 

. * f f lA - ? 1 

ln{ 1 • -.1 - N - —a* 

u v i 

t ; r x 4 

1 2 + 3 4 ^ i 

1 

(-l) t + l /E 


Not*: T&C scries for the last two generating functions can he found in most calculus hooks when power series are discussed. 


PDF created with pdfFactory trial version www.pdffactorv.com 




































344 


5 / Advanced Counting Techniques 


This follows since we obtain a term equal to a 1 1 in the product by picking'a term in the 
first sum a 1 " 1 , a term in the second sum ■, and a term in the third sum .v t \ where the 
exponents m, and ej satisfy the equation e\ -r^+ey - 17 and the given constraints. 

It is not hard to see that the coefficient of .v 17 in this product is 3. Hence, there 
are three solutions. (Note that the calculating of this coefficient involves about as much 
work as enumerating all the solutions of the equation with the given constraints. How ¬ 
ever, the method that this illustrates often can be used to solve wade classes of counting 
problems with special formulae, as we will see, Furthermore, a computer algebra sys¬ 
tem can be used to do such computations.) ■ 


EXAMPLE 11 In bow many different ways can eight identical cookies be. distributed among three dis¬ 

tinct children if each child receives at least two cookies and no more than four cookies? 

Solution: Since each child receives at feast two but no more than four cookies, for each 
child there is a factor equal to 

(A 2 + <’ + A 4 ) 

in the generating function for the sequence {rj, where c n is the number of ways to 
distribute n cookies. Since there are three children, this generating function is 

tv - r 4 + a 4 ) 3 . 

We need the coefficient of v- in this product. The reason is that the x s terms in the 
expansion correspond to the w ays that three terms can be selected, with one from each 
factor, that have exponents adding up to 8. Furthermore, the exponents of the term from 
the first, second, and third factors are the numbers of cookies the first, second, and third 
children receive, respectively. Computation shows that this coefficient equals 6. Hence, 
there are six ways to distribute the cookies so that each child receives at least two, but 
no mare than four, cookies. ■ 


EXAMPLE 12 Use generating functions to determine the number of ways to insen tokens worth $1, $2, 

and $5 into a vending machine to pay for an item that costs r dollars in both the cases 
when the order in which the tokens are inserted does not matter and when the order 
does matter. (For example, there are two ways to pay for an item that costs $3 when 
the order in which the tokens are inserted does not matter: inserting three $ l tokens or 
one $1 token and a $2 token. When the order matters, there are three ways: inserting 
three SI tokens, inserting a $1 token and then a $2 token, or inserting a $2 token and 
then a $1 token.) 

Solution: Consider the case when the order in which the tokens are inserted does not 
matter. Here, all we care about is the number of each token used to produce a total of r 
dollars. Since we can use any number of $ 1 tokens, any number of $2 tokens, and any 
number of $5 tokens, the answer is the coefficient of x r in the generating function 

( 1 4- A + r + A J + ■■■)(! + X~ - A' 4 + A 4 ’ 4- ■ ■ ■)( 1 4 A 5 + A' 10 4- A 15 - ■ ■ ■ ). 

( The first factor in this product represents the Si tokens used, the second the $2 tokens 
used, and the third the $5 tokens used.) For example, the number of ways to pay for 
an item costing $7 using $ I, $2, and $5 tokens is given by the coefficient of a 7 in this 
expansion, which equals 6. 
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EXAMPLE 13 


When the order in which the tokens are inserted matters, the number o!' ways to 
insert exactly n tokens to produce a total of r dollars is the coefficient of x r in 

(x + x 2 4 x 5 )", 

since each of the r tokens may be a $ 1 token, a $2 token, or a $5 token. Since any number 
of tokens may be inserted, the number of ways to produce r dollars using $ I, $2, or $5 
tokens, when the order in which the tokens are inserted matters, is the coefficient of 
x r in 


i + (X 4 JC- + X-) 4 (x + x 2 + A' 5 ) 2 


_l_ 

1 - (X 4 X 2 4 A 2 *) 

1 

1 - x — a 2 - ,r ? ' 


where we have added the number of ways to insert!) tokens, 1 token, 2 tokens, 3 tokens, 
and so on, and where we have used the identity 1/(1 - a) = 1 + x + r 2 t ■ with x 
replaced with x+x 2 4 a 5 . For example, the number of ways to pay for an item costing $7 
using $1, $2, and $5 tokens, when the order in which the tokens are used matters, is the 
coefficient of x 1 in this expansion, w hich equals 26. [To see that this coefficient equals 
26 requires the addition of the coefficients of a 7 in the expansions u +- x 2 - r x 5 ) k for 
f H 7, This can be done by hand with considerable computation, or a computer 
algebra system can be used.] g 


The following example shows the versatility of generating functions when used to 
solve problems with differing assumptions. 


Use generating functions to find the number of ^-combinations of a set with it elements. 
Assume that the binomial theorem has already been established. 

Solution: Each of the n elements in the set contributes the term (1 4 x) So the generating 
function fix) = Here fix) is the generating function forwhere a* 

represents the number of /.--combinations of a set with n elements. Hence, 

fix) = (1 + xf, 

But by the binomial theorem, we have 

/<,) = y f"U 

k =o W 


where 


Jet klin - it)! 


Hence, C(n, k), the number of ^-combinations of a set with n dements, is 
n\ 

ft: 
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EXAMPLE 14 


EXAMPLE 15 


Remark: We proved the binomial theorem in Section 4.3 using the formula for the num¬ 
ber of r-combinations of a set with n elements. This example shows that the binomial 
theorem, which can be proved by mathematical induction, can be used to derive the 
formula for the number of r-combinations of a set with n elements. 


Use generating functions to find the number of r-combinations from a set with n ele¬ 
ments when repetition of elements is allowed. 

Solution: Let G{x } be the generating function for the sequence {a r }, where a r equals 
the number of r-combinations of a set with n elements with repetitions allowed. That 
is, G{x) = ±,%t } a r x r . Since we can select any number of a particular member of the 
set with n elements when we form an r-combination with repetition allowed, each of 
the n elements contributes (1 + x + x 2 + ,c -i —) to a product expansion for G( x). Each 
element contributes this factor since it may be selected zero times, one time, two times, 
three times, and so on, when an r-combination is formed (with a total of r elements 
selected). Since there are n elements in the set and each contributes this same factor to 
GU), we have 

GU) = (1 +■ x + x 2 H-)". 


As long as |x| < 1, we have 1 + x + x 2 + ■ ■ ■ = 1/(1 - x), so 
GU) = 1/(1 - jO* = (1 - xy n . 

Applying the extended binomial theorem, it follows that 

(1 - x )-" = (1 + (-*))-" = 

The number of r-combinations of a set with n elements with repetitions allowed, when 
r a positive integer, is the coefficient a r of x f in this sum. Consequently, using Example 
8 we find that a r equals 

|(-l) r = (-lyCCn + r- 1, r) - (-ir 

= C(n + r - 1, r). ■ 



Note that this is the same result we stated as Theorem 2 in Section 4.6. 


Use generating functions to find the number of ’ways to select r objects of n different 
kinds if we must select at least one object of each kind. 

Solution: Since we need to select at least one object of each kind, each of the n kinds 
of objects contributes the factor (x + x 2 + x 3 + ■ ■ ■) to the generating function G(x) for 
the sequence {a,}, where a r is the number of ways to select r objects of /! different kinds 
if we need at least one object of each kind. Hence, 

G(X} ■= ( X + X 2 + ,t 3 + ■ ■ r - x"( 1 + x + X 2 +- -■ ■)" - x7( 1 - xf. 
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EXAMPLE 16 


Using the extended binomial theorem and Example 8, we have 
G(x) = .E7(l - a')" 

= x n ■ (i - xy n 



if 


= i\ x j : )' C(n -t r - l,rK-l)Y 

r — 0 


- 'V C( '7 + r — 1, r).v ,! ’ r 

r- [) 


= f C(r - 1. r - n)x J 

; - n 

= V C(r — 1, j— »)/. 


We have shifted the summation n the aext-to-last equality by setting i - n + r 
so that t = n when r = 0 and n 4- r - 1 = t - 1, and then we replaced t by r as the 
index of summation in the last equality to return to our original notation, Hence, there 
are C'(r - J, r - n) ways to select r objects of a different kinds if we must select at least 
one object of each kind. ■ 


USING GENERATING FUNCTIONS TO SOLVE RECURRENCE 
RELATIONS 


We can find the solution to a recurrence relation and its initial conditions by finding an 
explicit formula for the associated generating function. This is illustrated in the follow¬ 
ing examples. 


Solve the recurrence relation a k ~ 3a*. L fork = 1,2, 3,.. .and initial condition oq - 2. 

Solution: Let G(v) be the generating function for the sequence {a*}, that is, G(j«) = 
f/f o First note that 

AOU! = = Va*..,**, 

t-0 k ~1 

Using the recurrence relation, we see that 

GU) 3jtG( x) - V a t x k - 3 at -i.C 

i-fi k-‘ 

= + yjttk - V. ;.u 

k- I 
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since do = 2 and a k = 3a t -\, Thus, 

G(.v) - 3xG(x) — (1 - 3jc)G(j) = 2, 

Solving for Girl shows that G(x) = 2/(1 - 3.v). Using the identity 1/(1 - ox) = 
^i:_ (l a k x i: , from Table 1, we have 

GC.t) = 2^3^ ~ X 2 ' 3 *** 

k--t) k-L 0 

Consequently, a* = 2-3*. ■ 


EXAMPLE 17 Suppose that a valid code word is an fl-digit number in decimal notation containing an 

even number of Os. Let a ri denote the number of valid code words of length n. In Exam¬ 
ple 7 of Section 5.1 we showed that the sequence {a n } satisfies the recurrence relation 

a a = 8«„ _ i + lO" -1 

and the initial condition a\ — 9. Use generating functions to find an explicit formula 
for 

Solution To make our work with generating functions simpler, we extend this se¬ 
quence by setting on - 1; when we assign this value to 3y and use the recurrence 
relation, we have dj = 8 oq + 10° = 8+1 = 9, which is consistent with our original 
initial condition. {It also makes sense because there is one code word of length 0 —the 
empty string.) 

We multiply both sides of the recurrence relation by x” to obtain 
a„x" = 8a„-i-v fl + 1.0" -! at". 

Let G(x) = ‘ n n a n x” be the generating function of the sequence rvy, a\, aj .We 

sum both sides of the last equation starting w ith n = i, to find that 

G(.v) — 1 ■ ^ a„x" = 'y (Su,;- ] x’ 1 4 i 11'' ' si 

It — 1 B=I 

= 8 3 fln-i x n t y io n ’ 'x" 

o-l o-l 

= 8.vVa„ ,x" _1 4 rV Iff hr" -1 

rt = ! w - ] 

= sxy^.t- + jV iov 

o=0 o=0 

= SjtG(Jt) 4 r/(l - I O r). 

where we have used Example 5 to evaluate the second summation. Therefore, we have 
Givi - l = SvGivi + x!(\ - 10a), 

Solving for G(x) show's that 
rM _ — y 9x_ 

II X.r)(l lii.Vl 
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EXAMPLE 18 


Expanding the right-hand side of this equation into partial fractions (as is done in the 
integration of rational functions studied in calculus) gives 


C(jc) = - 


1 1 

+ 


2(1-8-* 1 - IOjt, 

Using Example 5 twice (once with a - 8 and once with a = JO) gives 


Gl r) = - [y 8V + V IQ'V 

3 h-0 


= y U& n + io"). 

n- 0 l 

Consequently, we have shown that 


•v. = - (8" + 10"). 


USING GENERATING FUNCTIONS TO PROVE IDENTITIES 

In Chapter 4 we saw how combinatorial identities could be established using combina¬ 
torial proofs. Here we will show that such identities, as well as identities for extended 
binomial coefficients, can be proved using generating functions. Sometimes the gener¬ 
ating function approach is simpler than other approaches, especially when it is simpler 
to work: with the closed form of a generating function than with the terms of the sequence 
themselves. We illustrate how generating functions can be used to prove identities with 
the following example. 


Use generating functions to show that 

V C(n, (:)' = C(2n, «) 
k =o 

whenever n is a positive integer, 

Solution: First note that by the binomial theorem C(2/t, n) is the coefficient of x n in 
(1 + a) 2 ". However, we also have 

(| + x) 3h = ((] + A )”] 2 

= \C(n, 0) -I- C(n, 1 )x -I- C(n, 2)a 2 + - ■ ■ + C(n, n)x n ] 2 . 

The coefficient of x' 1 in this expression is C{n, 0)C(n, n) + C(n, l)C(n, n — )) + 
Cin, 2 )C(n, n - 2) + ,.. + C(n, n)C(«,0). This equals 5y 0 Ci>i, k)\ since C{n, n - 
k) - C(n, k ). Since both C(2rt, n) and Y" ,__ 0 C(n, k) 2 represent the coefficient of x n in 
(I + a) 2 ", they must be equal. ■ 


Exercises 42 and 43 at the end of this section ask that Pascal’s identity and Van¬ 
dermonde’s identity be proved using generating functions. 
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Exercises 


1. Find the generating function for the finite sequence 2, 



g) U. 0 , 

h) 0.!), 0, 


2. Find the generating function for the finite sequence 1, 
4,16.64,256, 

3. Find a closed form for the generating function for each 
of the following sequences. (Assume a general form 
for the terms of the sequence, using the most obvious 
choice of such a sequence,) 

a) 0,2, 2.2. 2,2,2, 0,0, 0,0,0,,.. 

b) 0 , 0 , 0 , 1 , 1 , l, 1 . 1 , 1 ,... 

c) 0 . 1 , 0 , 0 , 1 , 0 . 0 , 1 , 0 , 0 , 1 ,... 
di 2,4, K, 16,32,64, 128,256,... 

'7\ m ji\ n\ 
lor 1 1 , 

n 2 , - 2 . 2 . - 2 .- 2 . : 

1, 1 , 1 , 1 , l, L... 

2, 3,4,... 

4. Find a closed form for the generating function for each 
of the following sequences. (Assume a general form 
for the terms of the sequence, using the most obvious 
choice a I such a sequence.) 

a) - 1 ,- 1 ,-l.-l, -I, -l.- 1 , 0 , 0 , 0 , 0 , 0 , 0 ,,.. 

b) I, 3, 9, 27, 81, 243, 729,... 
cl 0.0,3, -3.3, -3,3. -.1,... 
di 1 . 2 , 1 , 1 . 1 . 1 , 1 , 1 , 1 ,... 

e> f 7 j. 2 J f 7 ), 2 ; ('). 2' 1 ' 1 

[or [ i/ \zj 

f) -3,3, -3.3, - 3, 3,... 

g) 0 . 1 , -2,4,- 8 . 16. -32.64,.., 

h) 1 , 0 . 1 , 0 , 1 , 0 . 1 . 0 ,... 

5. Find a dosed form for the generating function for the 
sequence {s„) where 

a) a„ 5 for all n =0.1,2_ 

2 a for all n = 0 , 1,2 . 

2 for n - 3, 4, 5,.., and hq = = ax — 0, 

In + 3 for all n - 0, 1,2,.... 

•K\ 

" 0 , 1 , 2 ,.... 


. 0, 0,0, 0, 


b) ( 1 „ 

c) a„ ■ 

d) «j„ 


e) 


I for ail n 


f) a, -■ 


/« ■ 




for all n = 0 , 1 , 2 , 


Find a closed form for the generating function for the 
sequence {a,,} where 

a) - - ! for all n = 0, 1,2,... 

b) i 7 ,, - 2“ for n = 1,2, 3, 4,.,. and ao = 0, 

C) a„ -- n - I for n - 0. 1,2. 

d) a„ Min 4 ])! for n - 0, 1, 2. 

e) tt„ -= |” | font - 0, 1,2,,,.. 


f> a, = 


10 


for n - 0 , . 1 , 


7. For each of the following generating functions, provide 
a closed formula for the sequence it determines. 

a) (3x - 4) 3 

b ) + 1 Y 

c) 1/(1 - 5.x ) 

d) xV(l + 3x) 

e) * 2 + 3x + 7 + (1/(1 - x 2 )) 

f) (x 4 /() - x 4 )) - x 3 - ,v J - x - 1 

g) ■*--/(! - jr>- 

h) 2c :i 

8. For each of the following generating functions, provide 
a closed formula for the sequence it determines. 

a) (x 1 -t- 1 ) 3 

b) (3x If 

c) 1/(1 - 2 .r) 

d) r'(l - .r) :l 

e) < - 1 4 (1/(1 - 3.r)) 
fj (1 +.r-')/n 4 x) 3 

*g> xf{ l + x + r) 
h) e liJ - 1 

9. Find the coefficient of x 10 in the power series of each 
of the following functions. 

a) (1 4- .r 4 x 10 4 x 15 + ---) 3 

b) (x :i + X 4 4 X 5 + / + J 7 4 ■■■)-’ 

e) (x 4 + x 3 4 x^Xx 3 + v J 4x 5 4 ,r 6 4 x 7 )(! 4x + .r~ 4 
x 3 4 X 4 4 ■ ■ ■) 

d) for 2 4 .r 4 4 v 6 4 x 3 + ■ ■ fox 3 4 4 x 4 4 ■ - fox 4 r 

x 8 4x 12 + ■■■) 

e) (1 + ,r + ,r 4 4 x 6 + x 8 4 ■ ■■)( 1 4- X 4 + x s 4- r l: 4 
■■■HI 4 x 6 4x lj 4 r ls + *,-) 

II). Find the coefficient of x 9 in the power series of each of 
the following functions. 

a) (1 fr 1 4 x h + x> 4 - 

b) (x 2 4 .r 3 - x 4 4 x 5 4 x ft 4 ■ ■ -f 

c) (x 3 4 x 5 4- xfofo 3 4 x A )(x 4 x 2 + x 3 +■ .r 4 4 ■ ■ ■) 

d) (x 4 X 4 4 x’ 4 x m 4-)for 4 X 4 4 J 4, 4- ,r K 4- ■ ■ ■> 

el -x-.cf 

11. Find the coefficient of x 10 in the power series of each 
of the following functions. 

a) 1/(1 - 2 x) 

b) 1/(1 4 x f 

e) 1/(1 - x? 

d) 1/(14 2 x ) 4 

e) .-/(I - 3x} 3 

12. Find the coefficient of x 1 ’ in the power series of each 
of the following functions, 

a) 1/(1 4-3x) 

b) 1/(1 - 2x) 2 

c) 1/(1 + xf 

d) 1/(1 -4x ) 3 

e) x-Vn + 4x;r 
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13. Use generating functions to determine the number of 
different ways 10 identical balloons can be given to 
four children if each child receives at least two baJ- 
kions. 

14. Use generating functions to determine the number of 
different ways 12 identical action figures can be given 
to five children so that each child receives at most three 
action figures. 

15. Use generating functions to determine the number of 
different ways 15 identical stuffed animals can be 
given to six children so that each child receives at least 
one but no more than three stuffed animals. 

lft. Use generating functions to find the number of ways to 
choose a dozen bagels from three varieties—egg. salty, 
and plain—if at least two bagels of each kind but no 
more than three salty bagels ate chosen. 

17. In how many ways can 25 identical donuts be dis¬ 
tributed to four police officers so that each officer gets 
at least three but no more than seven donuts? 

18. Use generating functions to find the number of ways to 
select 14 balls from ajar containing l(X) red balls, 100 
blue balls, and 100 green balls so that no fewer than 3 
and no more than 10 blue balls are selected. Assume 
that the order in which the balls are drawn does not 
matter. 

19. What is the generating function for the sequence {cti, 
where o is the number of ways to make change for k 
dollars using $1 bills, $2 bills , $5 bills, and $10 bills? 

20. What is the generating function for the sequence {ct], 
where r* represents the number of ways to make 
change for k pesos using bills worth 10 pesos, 20 pesos, 
50 pesos, and 100 pesos? 

21. Give a combinatorial interpretation of the coefficient of 
a 4 in the expansion (I + jt + x 2 + x-' + ■ ■ -) 3 . Use this 
interpretation to find this number. 

22. Give a combinatorial interpretation of the coefficient of 
x 6 in the expansion (I + _t +- at 2 -r U + • ■ ■)". Use this 
interpretation to find this number. 

23. a) What is the, generating function for {a*}, where s* 

is the number of solutions of x\ + x 2 + x 2 = k when 
.ti, Ay, and xj are integers with jq a 2,0 ^ n s 
3, and 2 o, n e_ 5? ' 
hi Use your answer to part (a) to find (it,. 

24. a) What is file generating function for {at.}, where at 

is the number of solutions of ly + x% -+- x 3 + .< 4 = 
k when ,t i, xj, .vy, and .14 are integers with .nq - 
3, I 2 ; Xz -s 5,0 -s x.) < 4, and jl 4 > I? 
b> Use your answer to part (a) to find ay. 

25. Explain how generating functions can be used to find 
the number of ways in which postage of r cents cart be 
pasted on an envelope using 3-cent. 4-cent, and 20-cent 
stamps. 

a) Assume that the order the stamps are pasted on does 
not matter. 


b) Assume that the stamps are pasted in a row and the 
order in which they are pasted on matters. 

c) Use your answer to part (a) to determine the num¬ 
ber of ways 46 cents of postage can be pasted on an 
envelope using 3-cent, 4-eent, and 20-cent stamps 
when the order the stamps are pasted on does not 
matter. (Use of a computer algebra program is ad¬ 
vised.) 

d) Use your answer to part (b) to determine the num¬ 
ber of ways 46 cents of postage can be pasted in 
a row on an envelope using 3-cent, 4-cent, and 
20-cent stamps when the order in which the stamps 
are pasted on matters. (Use of a computer algebra 
program is advised.) 

26. a) Show that 1/(1 - x - i 2 - x i - x 4 - r' - x 6 ) is the 

generating function for the number of ways that the 
sum n can be obtained when a die is rolled repeat¬ 
edly and the order of the rolls matters, 

b) Use part (a) to find the number of ways to roll a to¬ 
tal of 8 when a die is rolled repeatedly, and the or¬ 
der of the rolls matters. (Use of a computer algebra 
package is advised.) 

27. Use generating functions (and a computer algebra 
package, if available) to find the number of ways to 
make change for $1 using 

a) dimes and quarters. 

b) nickels, dimes, and quarters 

c) pennies, dimes, and quarters. 

d) pennies, nickels, dimes, and quarters. 

28. Use generating functions (and a computer algebra 
package, if available) to find the number of ways to 
make change for $1 using pennies, nickels, dimes, and 
quarters with 

a) no more than 10 pennies, 
bj no more than If) pennies and no more than 10 nick¬ 
els. 

*c) no more than 10 coins, 

29. Use generating functions to find the number of ways to 
make change for $100 using 

a) $10, $20, and $50 bills. 

b) $5, $10, $20, and $50 bills, 

c) $5, S10, $20. and $50 bills if at least one bill of each 
denomination is used. 

d) .$5, $10, and $20 bills if at least one and no more 
than four of each denomination is used. 

30. IfG(r) is the generating function for the sequencefoj. 
what is the generating function for each of the follow¬ 
ing sequences? 

a) 2;io, 2fli, 2<3 2 , 2(ij,... b) 0 , o D , , a it ctj ,.,. 

c) 0, 0, 0, 0, a*, iii.. . d) 02. «j, «4.... 

e) 2(3;, iai, 4tJ 4 . ... {Hint: Calculus is required 
here.) 

f) rij), 2fl£jff], a 2 + Icwii. 2floo; + 2aia 2 , 2rio&i + 

2flitfj + u|_ 
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31. If Gix) is the generating function for the sequence {at}, 
what is the generating function for each of the follow¬ 
ing sequences? 

a) f), 0 , 0 , tix. 04 . as,. -. 

b) a-... 0. tii, 0, ti2, 0,.., 

c) 0, 0, 0, 0, no- hi. ffj. -.. 

d) ai;,. 2tii. 4«:, 8«s. 16^4, ,. - 

e) a 0 , a ; /3. a,/4.... (Him: Calculus is required 

here.) 

fl ill), Ui] x- £1 1, Uq -T n I -h til, U|j 4 til + Cl; 4- til, - ■ ■ 

32. Use generating functions to solve the recurrence rela¬ 
tion tii = la * i with the initial condition aa ~ 5. 

33. Use generating functions to solve the recurrence rela¬ 
tion <f( ~ lai | + 2 with the initial condition at, = I. 

34 L Ne generating functions to solve the recurrence re¬ 
lation at = liik-] + 4*' 1 with the initial condition 
aa = 1. 

35. Use generating functions to solve the recurrence rela¬ 
tion = 50(.._[ - 6u t . 3 with initial conditions n 0 = 6 
arid a , ■» 30. 

36. Use generating functions to solve the recurrence re¬ 
lation n, = at | r 2 + 2* with initial conditions 
% ■= 4 and ai ~ 12. 

37. Use generating functions to solve the recurrence rela¬ 
tion tit '- 4fj,i [ - 4«t 2 + k 2 with initial conditions 
ffu - 2 and a i = 5. 

38. Use generating functions to solve the recurrence rela¬ 
tion at -- 2(it i t t-4* + 6 with initial conditions 

(io - 20. cti 60. 

39. Use generating functions to find an explicit formula for 
the Fibonacci numbers. 

*40. a) Show that if n is a positive integer, then 

1/2 \ 

« /" r-4)-' 

b) Use the extended binomial theorem and part (aj to 
show that the coefficient of r" in the expansion of 
(1 - 4 jc>' 1,11 is P") for all non negative integers n, 
*41. (Calculus required) Let jC„} be the sequence of Cata¬ 
lan numbers, that is, the solution ut the recurrence re¬ 
lation Q =• i- with C n = Ci = 1 (see 

Example 8 in Section 5.1). 

a) Show that if (J(.r) is the generating function for the 
sequence of Catalan nu mbers, then tGi.rj 2 - (fi xt - 
1 - 0. Conclude (using the initial conditions) that 
(•<x: - (1 V T - 4x)/f2.t). 
h) Use F.xcrciSO 40 to conclude that 


( 


Guj =- V 


1 /2 rei 


■ that 


1 /2ft 
n +"l \ n 


42. Use generating functions to prove Pascal's identity: 
C(/t, r) =■ C(n - I, r) + C(n - 1, r - I) when n and 
rare posiiive integers with r < n. [Him: Use tire iden¬ 
tity (1 + j0 * = (1 + xf~ l + t-(l + xf 1 .] 

43. Use generating functions to prove Vandermonde s 
identity: C(m + n, r) = X^ 0 C(/n,f - k)C(n.k), 
whenever m. n, and r are nonnegative integers with 
r not exceeding either m or n. {Hint: Look at the 
coefficient of x r in both sides of (1 F x\ a ~" = 
(I + *)"(! + r)\l 

44. This exercise shows how to use generating functions to 
derive a formula for the sum of the first n squares. 

a) Show that ( x 1 + x)i( 1 - r ) 4 i s the generating function 

forthe sequence {a„}, where a„ = 1 3 + 2 2 h -fn 3 . 

b) Use part (a) to find an explicit formula for the sum 
l 3 + 2 1 -*-■■■+ rri. 

The exponential generating function for the sequence {«„} 
is the series 


For example, the exponential generating function for the se¬ 
quence L 1, 1.... is the function Y* =0 .*"/«! = e x . (You 
will find this particular series useful in the following exer¬ 
cises.) Note that c ' is the (ordinary ) generating function for 
the sequence 1,1, 1 / 21 , 1/31, 1 / 4 !. 

45. Find a dosed form for the exponential generating func¬ 
tion for the sequence {«„}, where 

a) o„ - 2 . b) a n - (- 1 )". 

c) a„ - 3". d) a„ - n + 1. 

e) & H = l-it! + l). 

46. Find a dosed form for the exponential generating func¬ 
tion for the sequence .i,}. where 

a) a„ - (- 2 )". b) a„ - -L. c) a„ - n, 

d) a„ = >t{n - 1 ). tt) a„ = i/((n+ l)(« + 2 )J. 

47. Find the sequence with each of the following functions 
as its exponential generating function, 

a) f(j r) - b) fix) - 3-r 1 

e) f{x) = c-‘ ~ 3p Ix 

d) fix) = (l - x) + e _Xr 

e) fix) = e -2 ' — ( 1/(1 

f) fix) = il - ,v) ‘ ■•l/;l - 2 t)) 

g) fix) = e * 1 

48. Find the sequence with each of the following functions 
as its exponential generating function. 

a) J\x) = e-' J b) f{x) = + , 

c) f[x) - e 4 -' r c- 4 -' d) fix) = (I -i- 2 x) -v f !l 

e) f(x) - a' -( 1/(1 + x)) 

f) fix) - xe* g) fix) ■ <•’' 

49. A coding system encodes messages using strings of oc¬ 
tal (base 8 ) digits. A codeword is considered valid if 
and only if it contains an even number of 7 s. 
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a) Find a linear non homogeneous recurrence relation 
for the number of valid codeword 1 ; of length n, What 
are the initial conditions? 

b) Solve this recurrence relation using Theorem 6 in 
Section 5.2. 

c) Solve this recurrence relation using generating 
functions. 

*50. A coding system encodes messages using strings of 
base 4 digits (that is, digits from the set {0,1, 2, 3}), A 
codew ord is valid if and only if it contains an even num¬ 
ber of Os and an even number of I s. Let a„ equal the 
number of valid codewords length n. Furthermore, let 
/>„, (■*, and d„ equal the number of strings of base 4 dig¬ 
its of length n w ith an even number of Os and an odd 
number of Is, with an odd number of Os and an even 
number of 1 s, and with an odd number of Os and an odd 
number of Is, respectively. 

al Show that ti« = 4" - u„ - b„ - r„ Use this to show 
that a „-1 = 2a n 4 b, i t c,„ £j„+i = b„ - c„ t-4", 
and tVi --- c„ - b„ + 4", 
h) What are«|,fo. C|, and d ]' 5 
r) Use parts (a) and (b) to find a>. 6 j. m, and a.. 

d) Use the recurrence relations in part (a), together 
with the initial conditions in part (b). to set up 
three equations relating the generating functions 
Air}. Bix), and C{x) for the sequences {a„f, •[/>„), 
and {t:,,}, respectively. 

c) Solve the system of equations front part (d) to get 
explicit formulae for A(r). B[ v), and C(x) and use 
these to get explicit formulae for a„, b„, c„. and d„. 

Generating functions are useful in studying the number 
of different types of partitions of an integer n. A parti¬ 
tion of a positive integer is a way to write this integer as 
the sum of positive integers where repetition is allowed 
and the order of the integers in the sum does not matter. 
For example, the partitions of 5 (with no restrictions) are 
1 + I t ! + 1 + 1. 1 v- I -t [ +• 2. 1 + 1 +- 3, 1 + 2 + 2. 

1 4-4*2 4 3. and 5, Exercises 51-56 illustrate some of these 
uses. 

51. S tew that the coefficient p{n) of x" in the formal power 
series expansion of 1 /((1 - .v)( 1 - x 2 ){! - ,r J ) ■ ■■) equals 
the number of partitions of ». 

52. Show that the coefficient p„(n) of x n in the formal 
power series expansion of 1/1(1 - ri( 1 j~)(1 - ,v 3 ) ■ ■ ■) 
equals the number of partitions of n into odd integers, 
that is, the number of ways to write n as the sum of odd 
positive integers, where the order does not matter and 
repetitions are allowed. 

53. Show that the coefficient Pd(n) of jf in the formal 
power series expansion of (I + .tj( I + U)(l + X } ) ■ ■ ■ 
equals the number of partitions of n into distinct parts, 
that is, the number of ways to write n as the sum of 


positive integers, where the order does not matter but 
no repetitions are allowed. 

54. Find p,jin ), the number of partitions of n into odd parts 
with repetitions allowed, and Pd{n), the number of par¬ 
titions of n into distinct parts, for i s n s 8 , by writ¬ 
ing each partition of each type for each integer. 

55. Show that if n is a positive integer, then the num¬ 
ber of partitions of n into distinct parts equals the 
number of partitions of n into odd parts with repe¬ 
titions allowed; that is, p„(n) = p,j(n}, [Hint: Show 
that the generating functions for p a (n) and pUn) are 
equal. | 

**56, (Calculus required) Use the generating function of p[n) 
to show that pin) - e c - : ' n for some constant C [Hardy 
and Ramanujan showed that p{n) ~ e’ r '-' 3 ' / 7(4 v /3n), 
which means that the ratio of pin) and the nght-hand 
side approaches I as n approaches infinity | 

Suppose that X is a random variable on a sample space S 

such that X(ij is a nonnegative integer for all s e $. The 

probability generating function for X is 

U'x(-t) = y piX(s) - k)x l: . 
t=f) 

57. (Calculus required) Show that if 6 > is the proba¬ 
bility generating function for a random variable X 
such that XU) is a nonnegative integer for ail s E S. 
then 

a) Gjrfi) = 1. b) ELY) - Cjjd), 
c) VfY) = C;(l) + Gj.fl) - 11 y. 

58. Let X be the random variable whose value is n if the 
first success. Occurs nn the nth trial when independent 
Bernoulli trials are performed, each with probability of 
success p. 

a) Find a closed formula for the probability generating 
function G x . 

b) Find the expected value and the variance of X using 
Exercise 57 and the closed form for the probability- 
generating function found in part fa). 

59, Let m be a positive integer. Let X„ be the random vari¬ 
able whose value is n if the mth success occurs on the 
(n 4 /filth trial when independent Bernoulli trials are 
performed, each with probability of success p. 

a) Using Exercise 44 in the Supplementary Exercises 
of Chapter 4, show that the probability generating 
function Gx m is given by G'(x)x m = p"/(! - qxU, 
where q = 1 - p. 

b) Find the expected value and the variance of X,„ us¬ 
ing Exercise 57 and the dosed form for the proba¬ 
bility generating function in part (a). 

60, Show that if X and Y are independent random vari¬ 
ables on a sample space S such that X(s) and YU) are 
nonnegativc integers for all j eS, then Gy+yU) - 
CxCJtjGyfx). 
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5.5 _ 

Inclusion-Exclusion 

INTRODUCTION 


A discrete mathematics class contains 30 women and 50 sophomores. How many stu¬ 
dents in the class are either women or sophomores? This question cannot be answered 
unless more information is provided. Adding the number of women in the class and 
the number of sophomores probably does not give the correct answer, because women 
sophomores are counted twice. This observation shows that the number of students in 
the class that are either sophomores or women is the sum of the number of women and 
the number of sophomores in the class minus the number of women sophomores. A 
technique for solving such counting problems was introduced in Section 4,1. In this 
section we will generalize the ideas introduced in that section to solve a wider range of 
counting problems. 


THE PRINCIPLE OF INCLUSION-EXCLUSION 


How many elements are in the union of two finite sets? In Section 1.5 it was shown that 
the number of elements in the union of the two sets A and B is the sum of the numbers 
of elements in the sets minus the number of elements in their intersection. That is, 

|A U B\ - |A'| + |B| - |A n B\, 

As we showed in Section 4.1, the formula for the number of elements in the union of 
two sets is useful in counting problems. The following examples provide additional 
illustrations of the usefulness of this formula. 


EXAMPLE 1 A discrete mathematics class contains 25 students majoring in computer science, 13 

students majoring in mathematics, and 8 joint mathematics and computer science ma¬ 
jors. How many students are in this class, if every student is majoring in mathematics, 
computer science, or both mathematics and computer science? 

Solution: Let A be the set of students in the class majoring in computer science and 
B be the set of students in the class majoring in mathematics. Then A f) B is the set 
of students in the class that are joint mathematics and computer science majors. Since 
every student in the class is majoring in either computer science or mathematics (or 
both), it follows that the number of students in the class is |A U B\, Therefore, 

|A U B\ = | A) +■ \B\ - 1 A n R\ 

= 25 +13-8 
- 30, 

Therefore, there arc 30 students in the class. This computation is illustrated in Fig¬ 
ure 1. ■ 


EXAMPLE 2 How many positive integers not exceeding 1000 are divisible by 7 nr 11? 
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EXAMPLE 3 


1 A U B\ = |A|+|fl| -|AnJS| *25 + 13-8 * 30 



Discrete Mathematics Class. 


|a U I = |^l|+|flj - |A rt Jt| = 142 + 90 - 1 1 - 220 



FIGURE 2 The Set of Positive Integers Not 
Exceeding 1000 Divisible by Either 7 or 11. 


Solution: Let A be the set of positive integers not exceeding 1000 that are divisible 
by 7, and let B be the set of positive integers not exceeding 10QO that are divisible 
by 11. Then A U Pis the set of integers not exceeding 1000 that are divisible by ei¬ 
ther 7 or 11, and .4 Hi B is the set of integers not exceeding 1000 that are divisible by 
both 7 and ] 1. From Example 2 of Section 2.3, we know that among the positive in¬ 
tegers not exceeding 1000 there are [1000/7J integers divisible by 7 and |1000/llJ 
divisible by 11. Since 7 and 11 are relatively prime, the integers divisible by both 
7 and 11 are those divisible by 7 * 11. Consequently, there are [ 1000/(1] ■ 7)J posi¬ 
tive integers not exceeding 1000 that are divisible by both 7 and IT It follows that 
there are 

>A U B\ = |,4| + \B\ - \A n B\ 

~ jiOOO I I 1000 _ j 1000 

”1 i \ + ["Ti J l7^TT_ 

= 142 + 90 - 12 
- 220 

positive integers not exceeding 1000 that are divisible by either 7 or 11. This compu¬ 
tation is illustrated in figure 2. ■ 


The next example shows how to find the number of elements in a finite universal 
set that are outside the union of two sets. 


Suppose that there are 1807 freshmen at your school. Of these, 453 are taking a course 
in computer science, 567 are taking a course in mathematics, and 299 are taking courses 
in both computer science and mathematics. How many axe not taking a course in either 
computer science or in mathematics? 

Solution: To find the number of freshmen who are not taking a course in cither math¬ 
ematics or computer science, subtract the number that are taking a course in either of 
these subjects from the total number of freshmen. Let .4 be the set of all freshmen tak¬ 
ing a course in computer science, and let B be the set of all freshmen taking a course in 
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\A OJJj = |/t|+|flj-jA n Bj = 45? + 567 -'229 = 721 


| .4 H fl| = 229 |t\ = l807 



FIGURE 3 The Set of Freshmen Not Taking a Course in Either Computer Science nr 
Mathematics. 


mathematics. It follows that |A| = 453, jftj = 567, and | A fl B\ = 299. The number of 
freshmen taking a course in either computer science or mathematics is 

\A U S| = |A + \B\ - |A n B\ = 453 + 567 - 299 = 721. 

Consequently, there are 1807 - 721 = 10S6 freshmen who are not taking a course in 
computer science or mathematics. This is illustrated in Figure 3. ■ 


Later in this section it will be shown how the number of elements in the union of 
a finite number of sets can be found. The result that will be developed is called the 
principle tif inclusion-exclusion. Before considering unions of n sets, where n is any 
positive integer, a formula for the number of elements in the union of three sets A, B, 
and C will be derived. To construct this formula note that |A| + |i? + |C| counts each 
dement that is in exactly one of the three sets once, elements that are in exactly two of 
the sets twice, and elements in all three sets three times. This is illustrated in the first 
panel in Figure 4. 



1,0 


Couu of elements bv 

N-itfi+ki 


(b) Co uni of elements by 

Ul+|if|+|c|" kna] 
MncHnnci 


(b) Count of elements by 
W+[flj+|C|-|j4nflj- 
lancl-lfrncl+[/tn end 


FIG t RE 4 Finding a Formula for the Number of Elements in the Union of Three Sets. 
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To remove the ovcrcount of elements in more than one of the sets, subtract the 
number of elements in the intersections of all pairs of the three sets, obtaining 

\A\ + I B\ ~ \C\ - \A n B\ - A n C| - B n c\. 

This expression .still counts elements t nai occur it! exactly one of the sets once. An 
element that occurs in exactly two of the sets is also counted exactly once, since this 
element will occur in one of the three intersections of sets taken two at a time. However, 
those elements that occur in all three sets will be counted zero times by tins expression, 
since they occur in all three intersections of sets taken two at a time. This is illustrated 
in the second panel in Figure 4. 

To remedy this undercount, add the number of elements in the intersection of all 
three sets. This final expression counts each element once, whether it is in one, two. or 
three of the sets. Thus, 

j.4 u b u c\ = \a\ + jfi| + c\ - |a n s| 'a n c\ - \b n r| t |,4 r a r, c . 

This formula is illustrated in the third panel of Figure 4. 

The following examples illustrate how this formula can be used. 


EXAMPLE 4 


A total of 1232 students have taken a course in Spanish, 879 have taken a course in 
French, and 114 have taken a course in Russian. Further, 103 have taken courses in 
both Spanish and French, 23 have taken courses in both Spanish and Russian, and 14 
have taken courses in both French and Russian. If 2092 students have taken at least one 
of Spanish, French, and Russian, how many students have taken a course in till three 
languages? 


Solution: Let 5 be the set of students who have taken a course in Spanish. F the set of 
students who have taken a course in French, and R the set of students who have taken 
a course in Russian. Then 


\S\ = 1232, 

V ' Fl - 103. 


and 


|F - 879, R - 114, 

■5n«| = 23. |Fn/?[-l4, 


[S U F U /?' = 2092 . 

Inserting these quantities into the equation 

\s u r u r\ - |sl - If I * |j?i - |5 n Fl - ]5 n r\ - ;/■ n j?i + )s n f n k\ 

gives 

2092 -■ 1232 + 879 * 114 -■ 103 - 23 - 14 r l,V n F n R\. 

Solving for |4 fl F fl R\ shows that S PI F fl R\ ~ 1. Therefore, there arc seven students 
who have taken courses in Spanish, French, and Russian. This is illustrated in Fig¬ 
ure 5. m 


We will now state and prove the inclusion-exclusion principle, which tells us how- 
many dements are in the union of a Unite number of finite sets. 
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THEOREM 1 


Isnfnfl]*? |snff=io3 



i /?| — 114 

|5UfUJ?| = 2W2 


FIGURE 5 The Set of Students Who Have Taken Courses in Spanish, French, and 
Russian. 


The Principle of ItKtesltat-^sdtKicm Let AiyA *.. „>Ll*fafce Mis. Then 

p.:< 

Uj 4 j u= y {A/|*- ?. 

lst*q ts 

+ 2 |A( n Aj n Ajtl - • • • + (-n^fA, n Aa n ••• n /i.l 

1 sK/Cisa 

Proof; We will prove the formula by showing that an dement in the union is counted 
exactly once by the right-hand side of the equation. Suppose that a is a member of 

exactly r of the sets Aj, A 2 . A n where 1 s r :£ n. This dement is counted C(r, 1) 

times by 2|A;J. It is counted C(r, 2) times by S|A, f) A tn general, it is counted C{r, rn) 
times by the summation involving mof the sets A,. Thus, this element is counted exactly 

C(r. 1) - C(r, 2) + C(r. 3}-+ (-J) r_L, C(r, r) 

times by the expression on the right-hand side of this equation. Our goal is to evaluate 
this quantity. From Theorem 7 of Section 4.3, we have 

C(r, 0) - C(r, I ) + C(>. 2) -.+ I I f O r. r) - a 

Hence, 

1 = CO; 0) = C(r, 1) - CO; 2) + ■ ■ ■ + (- - I) r+1 C(r, r). 

Therefore, each element in the union is counted exactly once by the expression on the 
right-hand side of the equation. This proves the principle of inclusion-exclusion. □ 

The inclusion-exclusion principle gives a formula for the number of elements in 
the union of n sets for every positive integer n. There are terms in this formula for the 
number of elements in the intersection of ever}' nonempty subset of the collection of 
the n sets. Hence, there are 2" - 1 terms in this formula. 
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EXAMPLE 5 Give a formula for the number of elements in the union of four sets. 

Solution: The inclusion-exclusion principle shows that 

jA[ u At U As U A 4 I — |Aj| + j.4j| 4 [Aj| + IA 4 I 

- \Ai n A 2 j - |A, n A3 1 - | A t n a 4 | - iA 2 n a 3 | - |a 2 n a 4 | - [As n a 4 | 
4 1 a 1 n a 2 n a 3 [ 4 |A] n a 2 n a 4 | + |A| n A3 n a 4 | 4 |a 2 n A3 n a 4 | 

- [a 1 n a 2 n a 3 n a 4 |. 

Note that this formula contains 15 different terms, one for each nonempty subset of 
{A], At, Aj, A 4 }. W 


Exercises 

1. How many elements are in A] J Ai if there are 12 el¬ 
ements in Aj, 18 elements in A 2 , and 

a) At n A: - 0 -1 

b) |A| nA;i = 1? 

C) |A, n A; I - 6 1 
d) At C A : ? 

2. There are 345 students at a college who have taken 
a course in calculus, 212 w ho have taken a course in 
discrete mathematics, and 188 who have taken courses 
in both calculus and discrete mathematics. How many 
students have laken a course in either calculus or dis¬ 
crete mathematics? 

3. A survey of households in the United States reveals that 
96% have at least one television set, 98% have tele¬ 
phone service, and 95% have telephone service and at 
least one television set. What percentage of households 
in the United Slates have neither telephone service nor 
a television set? 

4. A marketing report concerning personal computers 
states that 650,000 owners w ill buy a modem for their 
machines next year and 1.250,000 will buy at least one 
software package. If (he report states that 1,450,000 
owners will buy either a modem or at least one .soft¬ 
ware package, how many will buy both a modem and 
at least one software package 0 

5. Find the number of dements in A] UA.; U A? if there 
are 100 elements in each set if 

a) the sets arc pairwise disjoint. 

b) there are 50 common elements in each pair of sets 
and no elements in all three sets. 

c) there are 50 common elements in each pair of sets 
and 25 elements in all three .sets. 

d) the sets are equal. 

6. Find the number of dements in Ai U Ai 11 A; if there 
are 100 dements in A[, 1000 in Aj. and 10,000 in A 3 
sf 


a) A] C A 2 and Aj C Ay. 
h) the sets are pairwise disjoint, 

c) there are two elements common to each pair of sms 
and one element in all three sets. 

7, There are 2504 computer science students at a school. 
Of these, 1876 have taken a course in Pascal, 999 have 
taken a course in Fortran, and 345 have taken a course 
in C- Further, 876 have taken courses in both Pascal 
and Fortran, 231 have taken courses in both Fortran and 
C, and 290 have taken courses in both Pascal and C. 
If 189 of these students have taken courses in Fortran, 
Pascal, and C, how many of these 25(14 students have 
not taken a course in any of these three programming 
languages? 

8, In a survey of 270 college students, it is found that 
64 like Brussels sprouts, 94 like broccoli, 58 like 
cauliflower, 26 like both Brussels sprouts and broccoli, 
28 like both Brussels sprouts and cauliflower, 22 like 
both broccoli and cauliflower, and 14 like ail three veg¬ 
etables. How many of the 270 students do not like any 
of these vegetables? 

9, How many students are enrolled in a course either in 
calculus, discrete mathematics, data structures, or pro¬ 
gramming languages at a school if there are 507, 292. 
312, and 344 students in these, courses, respectively. 
14 in both calculus and data structures, 213 in both 
calculus and programming languages, 211 in both dis¬ 
crete mathematics and data structures, 43 in both dis¬ 
crete mathematics and programming languages, and no 
student may take calculus and discrete mathematics, 
or data structures and programming languages, con¬ 
currently? 

10. Find the number of positive integers not exceeding 100 
that are not divisible by 5 or by 7. 
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11. Find ihti number of positive Integers not exceeding 1 00 
that are either odd or the square of an integer. 

12. Find the number of positive integers not exceeding 
1000 that are either the square or the cube of an integer, 

13. How- many bit strings of length eight do not contain six 
consecutive Os? 

* 14. How many permutations of the 26 letters of the English 
alphabet do not contain any of the strings fish, rat, or 
bird! 

15. How man> permutations of the 10 digits either begin 
with the 3 digits 987. contain the digits 45 in the fifth 
and sixth positions, or end with the 3 digits 123? 

If), f low many dements are in the union of four sets if each 
Ofliw sets has 100 elements, each pairpfthe sets shares 
50 elements, cacti three of the sets share 25 elements, 
and there are 5 elements in ail four sets? 

17. How many elements arc in the union of four sets if 
the sets have 50, 60,70. and 80 elements, respectively, 
each pair of the sets has 5 elements in common, each 
triple of the sets has I common dement, and no element 
fs in all four sets'? 

18, How many terms are there in the formula for the num¬ 
ber df elements in the unton of 10 sets given by the 
principle of inclusion exclusion? 

19, Write out the explicit fomiula given by the principle of 
inclusion-exclusion for the number of dements in the 
union of live sets. 

20. How many elements are in the union of live sets if the 
sets contain 10.000 elements each, each pair of sets 
has 1000 common elements, each triple of sets has 100 


common elements, every four of the sets have 10 
common elements, and there is I clement in all live 
sets? 

21, Write out the explicit formula given by the principle of 
inclusion-exclusion for the number of elements in the 
union of six sets when iris known that no three of these 
sets have a common intersection. 

*22. Prove the principle of inclusion-exclusion using math¬ 
ematical induction. 

23. bet E]. Ei, and Ey be three events from a sample space 
5. Find a formula for the probability of ft, u Ez U F> 

24. Find the probability that when a coin is Hipped five 
times tails comes up exactly three times, the first and 
last flips come up tails, or the second and fourth flips 
come up heads, 

25. Find the probability that when four numbers from 1 to 
100, inclusive, are picked at random with no repetitions 
allowed, either all are odd all are divisible by 3, or all 
arc divisible by 5. 

26. Find a formula for the probability of the union of four 
events in a sample space if no three of them can occur 
at the same time, 

27. Find a formula for the probability of the union of five 
events in a sample space if no four of them can occur 
at the same time. 

28. Find a formula for the probability of the union of n 
events in a sample space when no two of these events 
can occur at the same time, 

29. Find a formula for the probability of the union of n 
events in a sample space. 


16 _ 

Applications of Inclusion-Exclusion 

INTRODUCTION 


Many counting problems can be solved using the principle of inclusion-exclusion. For 
instance, we can use this principle to find the number of primes less than a positive in¬ 
teger. Many problems can be solved by counting the number of onto functions from one 
finite set to another. The inclusion-exclusion principle can be used to find the number 
of such functions. The famous hateheek problem cun be solved using the principle of 
inclusion-exclusion. This problem asks for the probability that no person is given the 
correct hat back by a hateheek person who gives the hats back randomly. 

AN ALTERNATIVE FORM OF INCLUSION-EXCLUSION 


There is an alternative form of the principle of inclusion-exclusion that is useful in 
counting problems. In particular, this form can be used to solve problems that ask for 
the number of elements in a set that have none of n properties P\, P 2 . P„. 

Let A, be the subset containing the elements that have property P,. The number of 
elements with all the properties P h , P,, . p h will be denoted by N(P :i P h - ■ ■ P t .). 
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EXAMPLE 1 


Writing these quantities in terms of sets, we have 

\A i{ n A k n * - - n a (1 | - N(P i: p h ■ ■ ■ p h i 

If the number of elements with none of the properties P tl P 2 ,..., P„ is denoted by 
N{P\ P 2 - P' n ) and the number of elements in the set is denoted by N, it follows that 

N(P i Py ■ K) " ‘ v - ! '4i U A 2 U ■ ■ ■ U A„|, 

From the inclusion-exelusion pnncipie, we see that 

N(P\ Pj ■ ■ ■ P‘ n ) = N - y ;V( P1 1 + A'f P,P,) - V V,/',/' /• I 

l - , - rt \« i 1 %i<p:-ksn 

+ - (■ ■ IfjVfP, P n )_ 

The following example shows how the principle of inclusion-exclusion can be used to 
determine the number of solutions m integers of an equation with constraints. 


How many solutions does 
A] + A'j + Xj = 11 

have, where jq, x 2 , and * 3 are nonnegative integers with .r! < 3. x a es 4, andx 3 it 6 ? 

Solution: To apply the pnncipie of inclusion-exclusion, let a solution have property P\ 
is x\ > 3, property P 2 is ,v 2 > 4. and property P 3 is x 3 > 6 . The number of solutions 
satisfying the inequalities jc] st 3, x 2 < 4, and ai < 6 is 

S\P : P';Pp- = N - .V;7*i i -V. /*.; - V,/M t N(P t P 2 ) - \'iPJ\; + NiP 2 P ? ) 

-N<P } P : Py 

Using the same techniques as in Example 6 of Section 4.6, it follows that 

■ A' = total number of solutions - CT3 + 11 - 1, 11} = 78, 

■ ;V(/>i) = (number of solutions with x\ > 4) - COM - 1,7) - £7(9,7) - 36, 

■ AT-Pi) = (number of solutions with je 2 > 5) « C(3 + 6 - 1,6) = £7(8.6) - 28, 

■ NlPC) = (number of solutions with x 3 > 7) - £7(3+4 - 1.4) - £ 7 ( 6 , 4 ) ^ 13, 

■ NiP^Pj) - (number of solutions with a[ + 4and x-> si 1) - C (3 - 2 -12)- 

C (4,2) - 6, 

* N{P\P 3 ) = (num berof solutions with ,vi +- 4 and *3 ^ 7) - £7(3+0-1,0) - 1 . 

■ N{P 2 P 3 ) - (number of solutions with xj ■— 5 and x? > 7) - 0, 

■ N(P] PnP]) ~ (number of solutions with x, > 4, x 2 5, and tq +• 7 ) = (), 

Inserting these quantities into the formula for N(P\P' 2 P\) shows that the number of 
solutions with xq £ 3, xp 4, and xq ^ 6 equals 

N(P\P’ 2 P' 3 ) = 78 - 36 - 28 - 15 + 6 + 1 + 0- 0- 6 . ■ 


THE SIEVE OF ERATOST HENES 


The principle of inclusion-exclusion can he used to find the number of primes not ex¬ 
ceeding a specified positive integer. Recall that a composite integer is divisible by a 
prime not exceeding its square root. So, to find the number of primes not exceeding 
100, first note that composite integers not exceeding 100 must have a prime factor not 
exceeding 10. Because the only primes less than 10 are 2, 3, 5, and 7, the primes 
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not exceeding 500 are these four primes and those positive integers greater than 1 and 
not exceeding 100 that are divisible by none of 2, 3, 5, or 7. To apply the principle 
of inclusion-exclusion, let Pi be the property that an integer is divisible by 2 , let Pi 
be the property that an integer is divisible by 3, let P-i be the property that an integer 
is divisible by 5, and let P 4 be the property that an integer is divisible by 7. Thus, the 
number of primes not exceeding i GO is 

4 + atp; pwV). 

Since there tire 99 positive integers greater than 1 and not exceeding 100, the principle 
of inclusion-exclusion shows that 


MP[P' 2 P\P a ) = 99 - ATP,) - Ni F-,\ - N(P-i) - ATP 4 ) 

+ A(P,Pi) + JV(P, P 3 ) + A(P,P 4 ) + atp 2 p?) 

+ iV(P 2 P 4 ) + A(p,p 4 > 

- AtP P P. - - A'lPiPr/Ti - ,\(P| P-.Ps 1 - A/fP 2 P 3 P 4 ) 
+ A(P i PiP 1 P 4 ). 


The number of integers not exceeding 100 (and greater than 1) that are divisible by all 
the primes in a subset of {2, 3,5, 7} is [100/A'J, where N is the product of the primes 
in this subset. (This follows since any two of these primes have no common factor.) 
Consequently, 


A(p;p;p;p + ) = 99 - 


100 

100 

~2~J 



100 I 
5 


! 100 


, LOO 
+ 3 : 7 


| 100 
+ I —- 
(2-5 

I 100 I 

+ M 


100 

7 


, 1 100 
4 ! T-l 


\ 100 j 
i^J 


100 

100 I 

i 100 j 

100 

2 ■ 3 ■ 5 J 

[2-3*7] 

[2-5-7] 

3-5-7 


1 100 

■f- 

[ 2 ■ 3 * 5■7 

= 99 - 50 - 33 - 20 - 14+16 +10-1- 7 + 6 + 4 + 2 


-3-2-1-0-0 


- 21 . 

Hence, there are 4 + 21 - 25 primes not exceeding 100. 

The sieve of Eratosthenes is used to find all primes not exceeding a specified 
wel) positive integer. For instance, the following procedure is used to find the primes not 


Eratosthenes {276-I1M B.C.E.l, I, is known that Eratosthenes was bom in Cyrcnc, a tireek colony west 
web of E8ypt - antl spent lime swd > in £ a» Ptato’s Academy in Athens. We also knew that King Ptolemy II in¬ 
vited Eratosthenes to Alexandria to tutor his son and that later Eratosthenes became chief librarian at the 
famous library at Alexandria, a central repository of ancient wisdom. Eratosthenes was an extremely ver¬ 
satile scholar, writing on mathematics, geography, astronomy, history, philosophy, and literary criticism. 
Besides his work in mathematics, he is most noted for his chronology of ancient history and for his famous 
measurement of the size of the earth. 


PDF created with pdfFactory trial version www.pdffactorv.com 













5.6 Applications of Inclusion-Exclusion 363 


T4BI.E 1 The Sieve of Eratosthenes, 

Integers Divisible by 2 Other than 2 Receive 

Integers Divisible by 3 Other than 3 Receive 

an Underline 


an Underline 



12 3 4 5 6 7 8 9 

IQ 

1 

2 3 4 5 6 7 8 

9 

10 

11 12 13 14 15 16 17 18 19 

20 

1! 

12 13 ]4 15 j_6 17 18 

19 

20 

21 22 23 24 25 26 27 28 29 

30 

21 

22 23 24 25 26 27 28 

29 

30 

31 32 33 14 55 36 37 38 39 

40 

31 

32 33 34 35 36 37 38 

39 

40 

; 4! 42 43 44 45 46 47 48 49 

50 

41 

42 43 44 45 46 47 48 

49 

50 

5! 52 53 54 55 56 57 58 59 

60 

5] 

52 53 54 55 56 57 58 

59 

60 

! 61 62 63 64 65 66 67 68 69 

70 

61 

62 63 64 65 66 67 6S 

69 

70 

71 72 73 74 75 76 77 78 79 

80 

71 

72 73 74 75 76 77 78 

79 

so 

81 82 S3 84 85 86 87 88 89 

90 

81 

82 S3 84 85 S6 .87 88 

89 

90 

93 92 93 94 95 96 97 98 99 

100 

91 

92 93 94 95 96 97 98 

99 

100 

Integers Divisible by 5 Other than 5 Receive 

Integers Divisible by 7 Other than 7 Receive 

an Under line 


an 

Underline; Integers in Color Are Prime 

12 3 4 5 6 7 8 9 

.10 

! 

2 3 4 5 ft 7 8 

V 

10 

I ! [2 13 14 15 16 17 18 19 

20 

[[ 

12 13 14 15 16 17 18 

19 

20 

2) 22 23 24 |5 26 27 28 29 

30 

2f 

22 23 24 25 26 27 28 

29 

30 

31 32 33 34 35 36 37 38 39 

40 

31 

32 33 34 35 36 3 7 38 

39 

40 

41 42 43 44 45 46 47 48 49 

50 

41 

42 43 44 45 46 47 48 

49 

50 ! 

51 52 53 54 55 56 57 58 59 

60 

51 

52 53 54 55 .% 57 5 g 

59 

60 

61 62 63 64 65 66 67 68 69 

1 

61 

62 63 64 65 66 67 68 

69 

70 

71 72 73 74 75 76 77 78 79 

go 

7 1 

72 73 74 75 76 77 78 

79 

80 | 

Si 82 83 84 85 86 87 88 89 

90 

81 

82 83 84 85 86 87 88 

89 

90 

91 92 97 94 95 96 97 98 99 

1(1 

91 

92 93 94 95 96 9? qg 

99 

m 


exceeding 100. First the integers that are divisible by 2, other than 2, tire deleted. Since 
3 is the first integer greater than 2 that is left, all those integers divisible by 3, other 
than 3, are deleted. Since 5 is the next integer left after 3, those integers divisible by 
5, other than 5, are deleted. The next integer left is 7, so those integers divisible by 7, 
other than 7, are deleted Since all composite integers no! exceeding 11X1 are divisible 
by 2, 3, 5. or 7, all remaining integers except I are prime. In Table 1, the panels display 
those integers deleted at each stage, where each integer divisible by 2, other than 2, is 
underlined in the first panel, each integer divisible by 3, other than 3, is underlined in 
the second panel, each integer divisible by 5, other than 3, is underlined in the third 
panel, and each integer divisible by 7, other than 7, is underlined in the fourth panel. 
The integers not underlined are the primes not exceeding 100- 


THE NUMBER OF ONTO FUNCTIONS 


I he principle ol inclusion-exclusion can also be used to determine (he number of omo 
1 unctions from a set with m elements to a set with a elements. First consider the fol¬ 
lowing example. 
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EXAMP!,K 2 


THEOREM 1 


EXAMPLE 4 


How mans onto functions are there from a set with six elements to a set with three 
elements? 


Solution: Suppose that the elements in the codomain are b\, hi, and 63 , Let P\, /A. and 
P's be the properties that by, bj, and b } are not in the range of the function, respectively. 
N ote that a function is onto if and only if it has none of the properties P\, Pi, or P}. By 
the inclusion-exclusion principle it follows that the number of onto functions from a 
set with six elements to a set with three elements is 


.Vi= V - ( N( P -) +■ N(P 2 ) + A : Pi 1 1 

+ { N ( PyPi ) + N ( PyPi ) + Ai/':/Ml - A• Pj/M. 

where N is the total number of functions from a set with six elements to one with 
three elements. We will evaluate each of the terms on the right-hand side of this 
equation. 

From Example ft of Section 4.1, it follows that N = 3 & . Note that N(P, i is the 
number of functions that do not have b, in their range. Hence, there are two choices 
for the value of the function at each element of the domain. Therefore, N{P : ) = 2 6 , 
Furthermore, there are C(3, 1) terms of this kind. Note that N(PjPj) is the number of 
functions that do not have bj and bj in their range. Hence, there is only one choice for 
the value ofthe function at each elemented the domain. Therefore, N(PiPj) = l 6 = 1 . 
Furthermore, there are C( 3, 2) terms of this kind. Also, note that N(Py P 2 Py ) = 0, since 
this term is the number of functions that have none of h\, hi, and b-\ in their range. 
Clearly, there are no such functions. Therefore, the number of onto functions from a set 
with six elements to one with three dements is 

t ~ C(3. 1 ) 2 6 + C(3. 2)i 6 = 729 - 192 + 3 - 540. ■ 


The general result that tells us how many onto functions there are from a set with 
m elements to one with n elements will now be stated. The proof of this result is left as 
an exercise for the reader. 

Let m and n be positive integers with m a n. Then, these are 

n m - Cin, l)(n - l) m + C(n, 2X« - 2) m - ■ ■ • + (-l^CO. n - 1) * l m 
onto functions from a set with m elements to a set with n ele ment s , 

One of the many different applications of Theorem I will now be described. 


How many ways are there to assign five different jobs to four different employees if 
every employee is assigned at least one job? 


Solution: Consider the assignment of jobs as a function from the set of five jobs to the 
set of lour employees. An assignment where every employee gets at least one job is 
the same as ait onto function from the set of jobs to the set of employees. Hence, by 
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EXAMPLE 4 


EXAMPLE 5 


THEOREM 1 


Theorem 1 it follows that there are 

4-' - C(4, 1 )3 5 + C(4, 2)2 5 - C(4, 3) I s = 1024 - 972 + 192 - 4 = 240 
ways to assign the jobs so that each employee is assigned at least one job. ■ 


DERANGEMENTS 


The principle of inclusion-exclusion will be used to count the permutations of n objects 
that leave no objects in their original positions. Consider the following example, 


The Hatched; Problem A new employee checks the hats of h people at a restaurant, 
forgetting to put claim check numbers on the hats. When customers return for their hats, 
the checker gives them back hats chosen at random from the remaining hats. What is 
the probability that no one receives the correct hat? ■ 


Remark: The answer is the number of ways the hats can be arranged so that there is no 
hat in its original position divided by nl, the number of permutations of n hats. We will 
return to this example after we find the number of permutations of n objects that leave 
no objects in their original position, 

A derangement is a permutation of objects that leaves no object in its original position. 
web To solve the problem posed in Example 4 we will need to determine the number of 
derangements of a set of n objects. 


The permutation 21453 is a derangement of 12345 because no number is left m its orig¬ 
inal position. However. 21543 is not a derangement of 12345, because this permutation 
leaves 4 fixed. | 


Let D„ denote the number of derangements of n objects. For instance, D> = 2, 
since the derangements of 123 are 23! and 312. We will evaluate D„, for all positive 
integers n, using the principle of inclusion-exclusion. 

The number of derangements of a set with n elements is 

D --" ! [ 1 -n + 5rs + - +< - 1) “a]- 


Historical Note: tn rencontres (matches), an old French card game, the 52 cards tn a deck arc laid an in 
a row. The cards of a second deck are laid out with one card of the second deck on lop of each card of the 
first deck. The score is determined by counting the number of matching cards in the two decks. In ! 70S 
Pierre Raymond tie Montmort (1678 1719) posed Iv problems de rencontres: What is the probability that 
no matches lake place in the game of rencontres? The solution to Montmort’s problem is the probability that 
a randomly selected permutation of 52 objects is a derangement, namely. 0^/521. which, as we will sec. 
is approximately l/e. 
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Proof: Let a permutation have property P, if it fixes element i. The number of derange¬ 
ments is the number of permutations having none of the properties P, for i - 1,2,..., 
re, or 

l\ - mp[p’ 2 -p' a i 

Using the principle of inclusion -exclusion, it follows that 

o n = jv ' v ,v(f : )+y N[p t Pj) - y N(piPjP k ) 

P-'j 


+ ... , ( 1 r.'Vi/' /*_ 

where A f is the number of permutations of n elements. This equation states that the num¬ 
ber of permutations that fix no elements equals the total number of permutations, less 
the number that fix at least one element, plus the number that fix at least two elements, 
less the number that fix at least three elements, and so on. All the quantities that occur 
on the right-hand side of this equation will now be found. 

hirst, note that iV = re!, since N is simply the total number of permutations of re 
elements. Also, A (TV) = (re - ])!. This follows from the product rule, since N(P,) is 
the number of permutations that fix element i, so that the ith position of the permuta¬ 
tion is determined, but each of the remaining positions can be filled arbitrarily. Simi¬ 
larly, 

N(P,Pj) - (re - 2)!, 

since this is the number of permutations that fix elements i and j, but where the other 
re 2 elements can be arranged arbitrarily. In general, note that 

NiP u P h ---P im ) - (n - m)\. 

because this is the number of permutations that fix elements i lt i 2 . i„, but where the 

other re - m elements can be arranged arbitrarily. Because there are C(n, m) ways to 
choose rn dements from re, it follows that 

V \(/V> = C(n, Lire 1)!, 

! - n 


y A nPiPj) - C(re, 2 ){re ~ 2)!, 

i • i <j■ n 
and in general, 

*> . N(P, t Pi, ■ ■ • Pi m ) - C(n, m){n - m)l. 


Consequently, inserting these quantities into our formula for D,, gives 

0„ - re! - C{n, l)(re - 1)! + C(», 2)(» - 2)1-+ (- lfC(re, n)(n - re)! 


= re! - — 


-(re - 1)! + 


Lire - 1)V.2'<re 

.Simplifying this expression gives 




Ph = ret 


I! 2’ 


f - ir 
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TABLE 2 The Probability of a Derangement. 

n 

2 3 4 5 6 7 

D„in 

0.50000 0.33333 0.37500 0.36667 0.36806 0.36786 


It is now simple to find D a for a given positive integer n. For instance, using Theorem 
2 , it follows that 


D\ = 3! 



I 

21 


n 

3!] 



- 2 , 


as we have previously remarked. 

The solution of the problem in Example 4 can now be given. 


Solution: The probability that no one receives the correct hat is !)„in\. By Theorem 2, 
this probability is 


The values of this probability for 2 < n ^ 7 are displayed in Table 2. 

Using methods from calculus it can be shown that 

e- 1 = 1 ~^ + ~ -•■> + + 0.368. 

1 ! 2 ! n\ 

Since this is an alternating series with terms tending to zero, it follows that as n grows 
without bound, the probability that no one receives the correct hat converges to e 1 ~ 
0.368. In fact, this probability can be shown to be within 1 /(» + 1)1 of e' 1 . ■ 


Exercises 


L Suppose that in a bushel of 100 apples there are 20 that 
have worms in them and 15 that have bruises. Only 
those apples with neither worms nor bruises can be 
sold. If there are 10 bruised apples that have worms 
in them, how many of the 100 apples can be sold? 

2. Of 1000 applicants for a mountain-climbing trip in the 
Himalayas, 450 get altitude sickness, 622 are not in 
good enough shape, and 30 have allergies. An appli¬ 
cant qualifies if and only if this applicant does not get 
altitude sickness, is in good shape, and does not have 
allergies. If there are 111 applicants who get altitude 
sickness and arc not in good enough shape, 14 'who get 
altitude sickness and have allergies, 18 who are not in 
good enough shape and have allergies, and 9 who get 
altitude sickness, are not in good enough shape, and 
have allergies, how many applicants qualify? 

i. How many solutions does the equation j] + x 2 + x 2 - 
13 have where i|, at, and arc nonnegative integers 
less than 6? 

4. Find the number of solutions of the equation a [ + .t '2 
a-j -t ij - 17 where x„ i ~ 1. 2,3,4, are non negative 
integerssuchthat.ri s 3, A; 5 4,An * S.amLnt £ 8. 


5. Find the number of primes less than 200 using the prin¬ 
ciple of inclusion-exclusion. 

6 . An integer is called squarefree if it is not divisible by 
the square of a positive integer greater than 1. Find the 
number of squarefree positive integers less than 100. 

7. How many positive integers less than 10.000 are not 
the second or higher power of an integer ? 

8 . How many onto functions are there from a set with 
seven elements to one with five elements? 

9. How many ways are there to distribute six different 
toys to three different children such that each child gets 
at least one toy? 

10. In how many ways can eight distinct balls be dis¬ 
tributed into three distinct urns if each urn must con¬ 
tain at least one ball? 

11. In how many ways can seven different jobs be assigned 
to four different employees so that each employee is 
assigned at least one job and the most difficult job is 
assigned to the best employee? 

12. List all the derangements of {1,2,3,4}. 

13. How many derangements are there of a set wilh seven 
elements'.’ 
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14. What is the probability that none of 10 people receives 
[he correct hat if a hatched, person hands their hats 
hack randomly ! 

15 . A machine that inserts letters into envelopes goes hay¬ 
wire and inserts letters randomly into envelopes. What 
is the probability that in a group of 100 letters 

a) no letter is put into the correct envelope'' 

b) exactly 1 letter is put into the correct envelope? 

c) exactly 98 letters arc put into the correct envelope? 

d) exactly 99 letters are put into the correct envelope? 

e) all tetters arc pul into the correct envelope? 

16. A group of n students is assigned seats for each of two 
classes in the same cla$SKK3m. How many ways can 
these Xenix he assigned if no student is assigned the 
same seat for both classes? 

*17. Mow many ways can the digits 0, 1, 2. 3. 4, 5, 6. 7, 
8. 9 be arranged so that no even digit is in its original 

pOMlidi! : 

*18. I he :t combinatorial argument to show that the se¬ 
quence ■T),.}, where D„ denotes the number of de¬ 
rangements of » objects, satisfies the recurrence 
relation 

1)„ ■ itt - I it A, i ■) 

for n - 3. 


*19. Use Exercise ! 8 to show that 
D„ = nD„ M -lf 
for n — 1. 

20. Use Exercise 19 to find an explicit formula for /fi¬ 
ll. For which positive integers n. is D„, the number of de¬ 
rangements of tt objects, even? 

22. Suppose that p and q are distinct primes, Use the prin¬ 
ciple of inclusion-exclusion to find 4 n f •/ • the number 
of integers not exceeding pq that are relatively prune 
to pq 

*23, Use the principle of inclusion-exclusion to derive a for¬ 
mula for cM.fi) when the prime factorization of n is 

" " Pi'Pf'--P°4- 

*24. Show that if » is a positive integer, then 
ft! = Cfn, Q)D, F ( in. llD n _, 

F ■ ■ ■ - 1 - C7n, n - 1 )D\ + (irt, rt)Dq, 

where A; is the number of derangements oft objects. 

25. How many derangements of {1. 2, ?. 4, 5, 6} begin with 
the integers 1.2, find 3. in some order? 

26. How many derangements of {1, 2. 3.4, 5.6} end with 
the integers 1,2, and 3, in some order? 

27. Prove Theorem 1. 


Key Terms and Results 

TERMS 

recurrence relation; a formula expressing terms of a se¬ 
quence, except fur some initial terms, as a function of one 
or more previous terms of the sequence 
initial conditions; for a recurrence relation: the values of 
the terms of a sequence satisfying the recurrence relation 
before this relation takes effect 
linear homogeneous recurrence relation with constant 
coefficients; a recurrence relation that expresses the 
terms (it a sequence, except initial terms, as a linear com¬ 
bination of previous terms 

characteristic roots of a linear homogeneous recurrence 
relation with constant coefficients: the roots of the 
polynomial associated with a linear homogeneous recur¬ 
rence relation with constant coefficients 
linear mmhwnogencous recurrence relation with con¬ 
stant coefficients: a recurrence relation that expresses 
the terms of a sequence, except for initial terms, as a lin¬ 
ear combination of previous term*: plus a function which 
ts not identically zero that depends only on the index 
divide-und-vonquer algorithm: an algorithm that solves 
a problem recursively by splitting it into a fixed number 
ul smaller pn-hleins of the same type 


generating function of a sequence: the formal series that 
has the nth term of the sequence as the coefficient of a' 1 
sieve of Eratosthenes: a procedure for finding the primes 
less than a specified positive integer 
derangement: a permutation of objects such that no object 
is in its original place 

RESULTS 

The formula for the number of elements in the union of 
two finite sets: 

A LJ fi| = |A| + \Bi - |A fl B\ 

The formula for the number of elements in the union of 
three finite sets: 

\A uSucj = |,lj + |B| -t |C| - |A n B| - |A n c\ 

- \B n C| + |A hfin u; 

The principle of inclusion-exclusion: 

I'd; u u -■ ■ u a„J =■ y J4I - _y 1.4, n a,J 

+ y |Aj r\Aj n 4i| 

— + f-iy ,Tl, 4 l nun - n a„| 
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The number of onto functions from a set with m ele¬ 
ments to a set with n dements: 

rf- - On, 1K« - I r - C(n. 2)[k - IT 

-+ (-1)" l C(n,n -])-]“ 


The number of derangements of n objects: 

D '-“'[' _ Ti + 5!. + '-»^ 


Review Questions 


1. a) What is a recurrence relation? 

b> Find a recurrence relation for the amount of money 
that will be in an account after n years if $ 1,000,000 
is deposited in an account yielding 9%. 

2. Explain how the Fibonacci numbers are used to solve 
Fibonacci’s problem about rabbits. 

3. a) Find a recurrence relation for the number of steps 

needed to solve the Tower of Hanoi puzzle, 

b) Show how this recurrence relation can be solved us¬ 
ing iteration. 

4. a) Explain how to find a recurrence relation for the 

number of bit strings of length n not containing two 
consecutive Is. 

b) Describe another counting problem that has a solu¬ 
tion satisfying the same recurrence relation. 

5. Define a linear homogeneous recurrence relation of de¬ 
gree k. 

6. a) Explain how to solve linear homogeneous recur¬ 

rence relations of degree two. 

b) Solve the recurrence relation a„ - 13n,_i -22a„-i 
for n K 2 if fj(i = 3 and ny = 1 5. 

c) Solve the recurrence relation a„ = 14a„- [-49a F[ . j 
for n £: 2 if ao = 3 and c? L = 35. 

7. a) Explain how to find /( b k ) where k is a positive in¬ 

teger if /{rcl satisfies the divide-and-conquer recur¬ 
rence relation /(«) = af(nlb) + g(n) whenever b 
divides the positive integer n. 

b) Find/[25b) if/in) = 3/(n/4}f5«/4and/(l) = 

8. a) Derive a divide-and-conquer recurrence relation for 

the number of comparisons used to find a number in 
a list using a binary.' search. 

b) Give a big -O estimate for the number of compar¬ 
isons used by a binary' search from the divide-and- 
conquer recurrence relation you gave in (a) using 
Theorem I in Section 5.3. 

9. a) Give a formula for the number of elements in the 

union of three sets. 


b) Explain why this formula is valid. 

c) Explain how to use the formula from (a) to find the 
number of integers not exceeding 1000 that are di¬ 
visible by 6, 10. or 15. 

d) Explain how to use the formula from (a) to find the 
number of solutions in nonnegative integers to the 
equation uy + ty - xj + x 4 = 22 with < 8, .to < 
6, and jrj < 5- 

1(1. a) Give a formula for the number of elements in the 
union of four sets and explain why it is valid, 
b) Suppose the sets ,4), Ay, and A 4 each contain 25 
elements, the intersection of any two of these sets 
contains 5 elements, the intersection of any three of 
these sets contains 2 elements, and I element is in 
all four of the sets. How many dements are in the 
union of the four sets? 

11. a) State the principle of inclusion-exdu sion. 
b) Outline a proof of this principle. 

12. Explain how 1 the principle of inclusion-exclusion can 
be used to count the numher of onto functions front a 
set w ith m elements to a set with n elements. 

13. a) Flow can you count the numher of ways to assign 

m jobs to w employees so that each employee is as¬ 
signed at least one job? 

b) How many ways are there to assign seven jobs to 
three employees so that each employee is assigned 
at least one job? 

14. Explain bow the inclusion-exclusion principle can be 
used to count the number of primes not exceeding the 
positive integer n. 

15. a) Define a derangement, 

b) Why is counting the number of ways a hatcheck 
person can return hats to n people, so that no one 
receives the correct hat, the same as counting the 
number of derangements of n objects? 

e) Explain how to count the number of derangements 
of ft objects, 


S upplementary Exercises 


I. A group of 1(1 people begin a chain letter, with each per¬ 
son sending the letter to 4 other people. Each of these 
people sends the letter to 4 additional people, 


a) Find a recurrence relation for the number of letters 
sent at the nth stage of this chain letter, if no person 
ever receives more than one letter. 
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b) What ;ire the initial conditions for the recurrence re¬ 
lation in part ia 

c) How many letters are sent at the nth stage of the chain 
letter? 

2. A nuclear reactor has created 1X grams of a particular 
radioactive isotope. Every hour 1% of a radioactive iso¬ 
tope decays. 

a) Net tip a recurrence relation for the amount of this 
isotope left after it hours. 

hi What are the initial conditions for the recurrence re¬ 
lation in part (ah’ 
c) Solve this recurrence relation. 

3. Every hour the U.S. government prints 10,000 more $1 
bills, 4000 more $5 bills, 3000 more $10 bills, 2500 
more $20 bills, 1000 more $50 bills, arid the same num¬ 
ber of $100 bills as it did the previous hour. In the initial 
hour 1000 of each bill were produced, 

a) Set up a recurrence relation for the amount of money 
produced in the nth hour. 

hi What arc the initial conditions for the recurrence re¬ 
lation in pan (a)? 

c) Solve the recurrence relation for the amount of 
money produced in the nth hour. 

d) Set up a recurrence relation for the total amount of 
money produced in the first n hours. 

c) Solve the recurrence relation for the total amount 
of money produced in the first n hours. 

4. Suppose that every hour there are two new bacteria in a 
colony for each bacterium that was present the previous 
hour, and that all bacteria 2 hours old die. The colony 
starts with 100 new bacteria. 

a) Set up a recurrence relation for the number of bacte¬ 
ria present after n hours. 

hi What is the solution of this recurrence relation? 
c) When will the colony contain more than 1 million 
bacteria? 

5. Messages are sent over a communications channel using 
two different signals. One signal requires 2 microsec¬ 
onds for transmittal, and the other signal requires 3 mi¬ 
croseconds for transmittal. Each signal of a message is 
followed immediately by the next signal. 

a) Find a recurrence relation lor the number of different 
signals that can be sent in n micro seconds. 

b) What are the initial conditions of ihc recurrence re¬ 
lation m port (at? 

cl How many different messages can be sent in 12 mi¬ 
croseconds' 1 

6. A small post oflice has only 4-ecnr stamps, 6-cent 
.stamps, and 10-ccttt stamps. Find a recurrence relation 
for the [lumber of ways to form postage of n cents with 
these stamps it the order that the stamps are used mat¬ 
ters. What arc the initial conditions for this recurrence 
rdaiion? 

7. How many waysare there to form the following postages 
using the rules described in Exercise 6? 


a) 12 cents 

b) 14 cents 

c) 18 cents 

d) 22 cents 

X, Find the solutions of the simultaneous system of con¬ 
gruences 

U„ tl„ ] + b„ | 
b„ - it,, [ — b n | 
with a 0 = 1 and bo = 2, 

9, Solve the recurrence relation a„ = a\_ y fu r . -i if Vi ~ 

1 and ti] - 2. (Hint: Take logarithms of both sides to 
obtain a recurrence relation for the sequence log a„. 
n = 0,1,2. ) 

*10. Solve fhe recurrence relation o, = a 4 ( a 3 , if da - 2 
and £t| - 2. (See the hint for Exercise 9. i 

11. Find the solution of the recurrence relation a„ - 
3a* i -3ct„ i+a„ it I if no =2,0] = 4, and a 2 = K. 

12. Find the solution of the recurrence relation a„ -- 
3b, 1 - 3a„.j + if jq -- 2, tq -■ 2, and «i 3 - 4. 

*13. Suppose that in Example 4 of Section 5.1 a pair of rab¬ 
bits leaves the island after reproducing twice. Find a 
recurrence relation for the number of rabbits on the is¬ 
land in the middle of the nth month, 

14. Find the solution to the recurrence relation fin) =- 

3/(u/5) 2u 4 , when n is divisible by 5. for n = 5\ 

where k is a positive integer and /(1) = 1. 

15. Estimate the sine of/ in Exercise 14 if/ is an increasing 
function. 

Hu Find a recurrence relation that describes the number of 
comparisons used by the following algorithm: Find the 
larges! and second largest elements of a sequence of » 
numbers recursively by splitting the sequence into two 
.subsequences with an equal number of terms, 01 where 
there is one more term in one subsequence than in the 
other, at each stage. Stop when subsequences with two 
terms are reached. 

17. Estimate the number of comparisons used by the algo¬ 
rithm described in Exercise 16, 

Let {a a \ he a sequence of real numbers. The forward dif¬ 
ferences of ibis sequence are defined recursively as fol¬ 
lows: The first forward difference is Ait„ = u r|fl - a„: 

the {}: - 1 )th forward difference A* +t «„ is obtained from 

A Vs by A* + V - AV + ! _ A Vi- 

18. FindA«„ where 
a) «„ - 3. 

bi a„ - 4ti t ?. 

c) a„ -■ tr -r n -l- 1. 

19. Let a„ = 3n 3 + n -I 2, Find AV„ where k equals 
a) 2. b) 3. c> 4. 

*20. Suppose that a„ - P{n ] where P is a polynomial of 
degree d. Prove that A ,3-1 o„ = 0 for all nonnegalive 
integers 11 . 
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21. I .el and {b,S be sequences of real numbers. Show 

that 

Ai<;,A r ) ■= 

22. Show that if F(x) and C/(,.v) are the generating functions 
For the sequences {Pt? and {(>.,,}. respectively, and e and 
if are real numbers, then U'F l JG).x> is the generating 
function for k-.s, + dh k \. 

23. (Calculus required) This exercise shows how generat¬ 
ing functions can be used tu solve the recurrence rela¬ 
tion in t- 1 iu, ;J i ■ u„ + I l/u!i for n A 0 with initial 
Condition up 1. 

a) Let til r! be the generating function fur h/„f. Show 
thai G U) : G'j.v) - C and G'(0| = 1. 
lit Show from part (a) that (e r C(A')!' - I, and con¬ 
clude that Q.ci -- vc T i- /*■'. 
cl Use pari (b) to find a closed form for a„. 

24. Suppose that 14 students get an A on the first exam in 
a discrete mathematics class, and IS get an A on the 
second exam. If 22 students received an A on either 
the first exam or the second exam, how many students 
received an A on both exams? 

25. There are 323 farms in Monmouth County that have at 
least one of horses, cows, and sheep. If 224 have horses, 
85 have cows, 57 have sheep, and 18 farms have ail 
three types of animats, how many farms have exactly 
two of these three types of’animals? 

26. Queries to a database of student records at a college 
produced the following data: There are 2175 students 
at the college, 1675 of these are not freshmen, 1074 
students have taken a course in calculus. 444 students 
have taken a course in discrete mathematics, 607 stu¬ 
dents are not freshmen and have taken calculus, 350 
students have taken calculus and discrete muthenuu- 
ics. 201 students arc not freshmen and have taken dis¬ 
crete mathematics, ansi 143 students are not freshmen 
and have taken both calculus and discrete mathematics. 
Can all the responses to the queries he Correct? 

27. Students in the school ot mathematics at a umver 
sity major in one or more of the following four areas: 


applied mathematics (AM), pure mathematics (PM), 
operations research (OR), and computer science (CS), 
How many students are in this school if (including joint 
majors) there are 23 students majoring in AM; 17 stu¬ 
dents majoring in PM; 44 in OR; 63 in CS; 5 in AM 
and PM; 8 in AM and CS; 4 in AM and OR; 6 in PM 
and CS; 5 in PM and OR; 14 in OR and CS; 2 in PM, 
OR, and CS; 2 in AM. OR, and CS; 1 in PM, AM, and 
OR; 1 in PM. AM, and CS; and 1 in all four fields 

28. How many terms are needed when the inclusion- 
exclusion principle is used to express the number of 
elements in the union of seven sets if no more than five 
uf these sets have a common dement? 

29. How many solutions in positive integers are there to the 
equation .ij -t-jej + jj = 20 with 2 < .ty <6,6 < x 2 < 
10, andfl < *3 < 5? 

30. How many positive integers less than 1,000,000 are 

a) divisible by 2, 3, or 5? 

b) not divisible by 7, II, or 13? 
cl divisible by 3 but not by 7? 

31- How many positive integers less than 200 are 
a) second or higher powers of integers? 
bl either second or higher powers of integers or 
primes? 

C) not divisible by the square of an integer greater 
than 1 1 

dl not divisible by the cube of an integer greater 
than 1 ? 

e) not divisible hy three or more primes? 

*32. How many ways are there to assign six different jobs to 
three different employees if the hardest job is assigned 
to the most experienced employee and the easiest job 
is assigned to the least experienced employee? 

33- What is the probability that exactly one person is given 
back the correct hat by a hatcheck person who gives n 
people their hats back at random? 

34. How many bit strings of length six do not contain four 
consecutive Is? 

35. What is the probability that a bit string of length six 
contains at least four Is? 


Computer Projects 

WRIT!' PkUukAMS \vn I I THP ft>1 .LOWING INPUT AND OUTPUT. 


1. Given .i positive integer«. list all the moves required in 
the Tower of Hanoi puzzle to move n disks from one peg 
to another according to the rules of the puzzle. 

2 Given a positive integer n and an integer k with i < k s. 
n. list all the moves used by the Frame-Stewart algorithm 
! described i n the preamble to Exercise 48 of Section 5,1) 
to mine « disks from one peg to another using four pegs 
according to the rules of the puzzle. 


3. Given a positive integer n. list all the bit sequences of 
length n that do not have a pair of consecutive Os. 

4. Given a positive integer n, write out all ways to paren¬ 
thesize the product of n 1 1 variables. 

5. Given a recurrence relation ct„ = tya,. ] + ctPn-i where 
C: and i '2 are real numbers, initial conditions an = Cp 
and «| =- Ci, and a positive integer k, find a* using it¬ 
eration , 
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6, Given a recurrence relation a n = 4- cjo n 2 and 

initial conditions £i(i = Co and a 1 = C]. determine the 
unique solution. 

7, Given a recurrence relation of the form fin I = 
etf(nlh) + v, where a is a real number, b is a pos¬ 
itive integer, and c is a real number, and a positive 
integer k, find /(ft*) using iteration. 

8, Given the number of elements in the intersection of 
three sets, the number of elements in each pairwise 


intersection of these sets, and the number of dements 
in each set, find the number of elements in their union. 

9. Given a positive integer n, produce the formula for the 
number of elements in the union ot n sets. 

10 , Given positive integers m and n. find the number of 
onto functions from a set with m elements to a set with 
n elements. 

11. Given a positive integer n. list all the derangements of 

the set {1.2,3_«}. 


Computations and Explorations 

I'.St A COMPUTATIONAL PROGRAM OR PROGRAMS YOU HAVE WRITTEN TO DO THE FOLLOWING EXERCISES. 


1. Find the exact value of /mo, fix, and /moo where f„ is 
(he nth Fibonacci number. 

2. Find the smallest Fibonacci number greater than 
1,000,(TO, greater than 1,000,000,000. and greater 
than 1,000.000,000,000. 

3. Find as many prime Fibonacci numbers as you can. It 
is unknown whether there are infinitely many of these. 

4. Write out all the moves required to solve the Tower of 
Hanoi puzzle with 10 disks, 

5. Write out all the moves required to use the Frame- 
Stewart algorithm to move 20 disks from one peg to 
another peg using four pegs according to the rules of 
the Revc's puzzle. 

6. Verify the Frame conjecture for solving the Reve’s puz¬ 
zle for n disks for as many integers n as possible by 
showing that the puzzle cannot be solved using fewer 
moves than are made by the Frame-Stewart algorithm 
with the optimal choice of k. 

7. Compute the number of operations required to mul¬ 
tiply two integers with n bits for various integers n 


including 16, 64, 256, and 1024 using the fast mul¬ 
tiplication described in Section 5.3 and the standard 
algorithm for multiplying integers (Algorithm 4 in 
Section 2.4). 

8, Compute the number of operations required to multi¬ 
ply two n x n matrices for various integers n including 
4, 16, 64, and 128 using the fast matrix multipli¬ 
cation described in Section 5.3 and the standard 
algorithm for multiplying matrices (Algorithm 1 in 
Section 2.6), 

9. Use the sieve of Eratosthenes to find all the primes not 
exceeding 1000. 

10 . Find the number of primes not exceeding H),(M)G using 
the method described in Section 5,6 to find the number 
of primes not exceeding 100. 

11 . List all the derangements of) I, 2, 3, 4,5,6, 7. 8}. 

12 . Compute the probability that a permutation of n objects 
is a derangement for all positive integers not exceeding 
20 and determine how quickly these probabilities ap¬ 
proach the number e. 


Writing Projects 

RESPOND l'OTHF FOLLOW IXO WITH ESSAYS . Si NO OUTSIDE SOLRCRS. 


1, Find rhe original source where Fibonacci presented his 
puzzle about modeling rabbit populations, Discuss this 
problem and other problems posed by Fihonacci and 
give some information about Fibonacci himself, 

2, Explain how the Fibonacci numbers arise in a variety 
of applications, such as in phyllotaxis, the study of ar¬ 
rangement of leaves in plants, in the study of reflections 
by mirrors, and so on. 

3, Describe diffeTenl variations of the Tower of Hanoi puz¬ 
zle, including those with more than three pegs (including 
the Revc's puzzle discussed in the text and exercises), 
those where disk moves are restricted, and those where 


disks may have the same size. Include what Is known 
about the number of moves required hi solve each varia- 
liun, 

4 . Discuss as many different problems as possible where 
the Catalan numbers arise. 

5. Look up the definition of the lucky numbers. Explain 
how they are found using a sieve technique similar to 
the sieve of Eratosthenes. Find all the lucky numbers 
less than I (TO 

6. Describe how sieve methods are used in number theory, 
What kind of results have been established using such 
methods',' 
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7. Look up the roles of the old French card game of rencon - 
ms. Describe these rules and describe the work of Pierre 
Raymond de Montmort on k problitne de rencontres. 

8 . Describe how exponential generating functions can be 
used to solve a variety of counting problems. 

9. Describe the Polya theory of counting and the kind of 
counting problems that can be solved using this theory'. 


10. The prnblente des menaces (the problem of the house¬ 
holds) asks for the number of ways to arrange n couples 
around a table so that the sexes alternate and no hus¬ 
band and wife are seated together. Explain the method 
used by E. Lucas to solve this problem. 

11. Explain how wok polynomials can be used to solve 
counting problems. 
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Relations 


R elationships between elements of sets occur in many contexts. Every' day we 
deal with relationships such as those between a business and its telephone num¬ 
ber, an employee and his or her salary, a person and a relative, and so on. In 
mathematics we study relationships such as those between a positive integer and one 
that it divides, an integer and one that it is congruent to modulo 5, a real number and 
one that is larger than it, and so on. Relationships such as that between a program and 
a variable it uses and that between a computer language and a valid statement in this 
language often arise in computer science. 

Relationships between elements of sets are represented using the structure called 
a relation. Relations can be used to solve problems such as determining which pairs 
Of cities are linked by airline flights in a network, finding a viable order for the differ¬ 
ent phases of a complicated project, or producing a useful way to store information in 
computer databases. 


6.1 


Relations and Their Properties 


INTRODUCTION 


The most direct way to express a relationship between elements of two sets is to use 
ordered pairs made up of two related elements, For this reason, sets of ordered pairs 
are called binary relations. In this section we introduce the basic terminology used to 
describe binary relations. Later in this chapter we will use relations to solve problems 
involving communications networks, project scheduling, and identifying elements in 
sets with common properties. 

DEFINITION 1. Let A and B be sets. A binary relation from A to B is a subset 
of A >( B. 

In other words, a binary relation from A to B is a set R of ordered pairs where the 
first element of each ordered pair comes from A and the second element comes from B. 
We use the notation aRb to denote that in, h) G R and uRh to denote that [a, b) G R. 
Moreover, when (a, b) belongs to R, n is said to be related to h by R. 

Binary relations represent relationships between the elements of two sets. We will 
introduce n-ary relations, which express relationships among elements of more than 
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EXAMPLE 1 


EXAMPLE Z 


EXAMPLE 3 


two sets, later in this chapter. We will omit the word binary when there is no danger of 
confusion. 

The following are examples ofreiations. 


Let A be the set of students in your school, and let B be the set of courses. Let R be the 
relation that consists of those pairs (a, b) where a is a student enrolled in course b. For 
instance, if Jason Goodfriend and Deborah Sherman are enrolled in CS518, which is 
Discrete Mathematics, the pairs (Jason Goodfriend, CS518) and (Deborah Sherman, 
CS518) belong to R. If Jason Goodfriend is also enrolled in CSS (0, which is Data 
Structures, then the pair (Jason Goodfriend, CS510) is also in R. However, if Debo¬ 
rah Sherman is not enrolled in CSS 10, then the pair (Deborah Sherman, CS510) is not 
in R, I 


Let A be the set of all cities, and let B be the set of the 50 states in the United States 
of America. De/ine the relation R by specifying that (a, h) belongs to R if city a is in 
state b. For instance, (Boulder, Colorado), (Bangor, Maine), (Ann Arbor, Michigan), 
(Cupertino, California), and (Red Bank, New Jersey) are in R. ■ 


Let A = {0,1, 2) and B = {«, b}. Then {(0, a), (fJ. b). (I, a\ (2, /?)} is a relation from A 
to B. This means, for instance, that OR a, but that \Kb. Relations can be represented 
graphically, as shown in Figure 1, using arrows to represent ordered pairs. Another way 
to represent this relation is to use a table, which is also done in Figure 1. We will discuss 
representations of relations in more detail in Section 6.3. ■ 

FUNCTIONS AS RELATIONS 


Recall that a function / from a set A to a set B (as defined in Section 1.6) assigns a 
unique element of B to each element of A. The graph of / is the set of ordered pairs 
(a, b) such that b = f(a). Since the graph off is a subset of A X B, it is a relation from 
A io B. Moreover, the graph of a function has the property that every element of A is 
the first element of exactly one ordered pair of the graph. 

Conversely, if R is a relation from A to B such that every element in A is the first 
dement of exactly one ordered pair of R, then a function can be defined with R as its 
graph. This can be done by assigning to an element a of A the unique dement b £ B 
such that (c. b) G R. 



FIGURE 1 Displaying the Ordered Pairs in the Relation R from Example 3. 
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EXAMPLE 4 


EXAMPLES 


A relation can be used to express a one-to-many relationship between the elements 
of the sets A and B, where an element of A may be related to more than one element 
of B. A function represents a relation w here exactly one element of B is related to each 
element of A. 

RELATIONS ON A SET 


Relations from a set A to itself are of special interest. 

DEFINITION % A relation cm the set A is a relation from A to A. 

In other words, a relation on a set A is a subset of A x A. 


Let A be the set {1,2,3,4}. Which ordered pairs are in the relation R = {(«, ft) \ 
a divides ft}? 

Solution: Since (a, ft) is in R if and only if a and b are positive integers not exceeding 
4 such that a divides ft, we see that 

R = {(1, 1), (1, 2), fl, 3), (1,4), (2, 2), (2,4), (3, 3), (4, 4)}, 

The pairs in this relation are displayed both graphically and in tabular form in 
Figure 2. ■ 

Next, some examples of relations on the set of integers will be given. 

Consider the following relations on the set of integers: 

R\ = {(a, ft 1 1 a i ft} 

R> - {(", ft); a > ft} 

R\ = {(a, ft) | a = ft or a = -ft} 

Ri = {(fit, ft) | a = ft} 

ft; - {(£1, ft) 1 a = ft + 1} 

Rt = {(«, ft) i a 4- ft £ 3}. 

Which of these relations contain each of the pairs (1, 1), (1, 2), (2,1),(1, — I), and (2, 2)? 



FIGURE 2 Displaying the Ordered Pairs in the Relation R from Example 4. 


PDF created with pdfFactory trial version www.pdffactorv.com 













6.1 Relations and Their Properties 377 


EXAMPLE 6 


EXAMPLE 7 


EXAMPLES 


Remark: Unlike the relations in Examples 1-4, these are relations on an infinite set. 

Solution: The pair {1, 1} is in j? u /J 4 , and (1,2) is in R\ and R u . (2, 1) is in R 2 . 
/fs, and R(„ (I, -1) is in AS, A 3 , and Af,; and finally, (2, 2) is in A|, A 3 , and A 4 . ■ 


It is not hard lo determine the number of relations on a finite set, since a relation 
on a set A is simply a subset of A X A. 


How many relations are there on a set with n elements ' 1 

Solution: A relation on a set A is a subset of A x A, Since A x A has rr elements when 
A has n elements, and a set with m elements has 2 m subsets, there are 2" J subsets of 
A x A. Thus, there are 2"' relations on a set with n elements, ■ 

PROPERTIES OF RELATIONS 


There are several properties that are used to classify relations on a set. We will introduce 
the most important of these here. 

In some relations an element is always related to itself, For instance, let R be the 
relation on the set of all people consisting of pairs (jt, y) where a and y have the same 
mother and the same father. Then . 1 R x for every person x. 

DEFINITION 3. A relation R on a set A is called reflexive if (a, a) £ R for every 
element a G A. 


We see that a relation on A is reflexive if ever}' elemenl of A is related to itself. The 
following examples illustrate the concept of a reflexive relation. 


Consider the following relations on {1, 2,3, 4}: 

R t = {(1, 11,(1, 2), (2. I), (2,2). (3.4), (4,1), (4, 4)} 

/?: ={(U), (I, 2), (2.1)} 

R ? = {Cl. 1 HI, 2). (1, 4), (2, I), (2,2), (3.3). (4, 1), (4,4)} 

*4 = {(2,4(3. 1). (3,2), (4,1), (4, 2), (4,3)} 

R 5 - {(1. 0,(1, 2), (1, 3), (1,4), (2, 2). (2,3), (2, 4), (3,3), (3. 4), (4,4)} 

R, - {(3,4)}. 

Which of these relations are reflexive 1 

Solution: The relations A3 and R-, are reflexive since they both contain all pairs of the 
form (£z, t 7), namely, (I, I), (2, 2), (3, 3), and (4,4), The other relations are not reflexive 
since they do not contain all of these ordered pairs. In particular, R ], AS. R 4 , and A f - are 
not reflexive since (3,3) is not in any of these relations. ■ 

Which of the relations from Example 3 are reflexive? 
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EXAMPLE 9 


EXAMPLE 10 


EXAMPLE 11 


Solution: The reflexive relations from this example are R\ I since u x a for every 
integer a), Rj. and ff 4 . For each of the other relations in this example it is easy to 
find a pair of the form (a, a) that is not in the relation. (This is left as an exercise for 
the reader.) ■ 


Is the "divides" relation on the set of positive integers reflexive? 

Solution: Since a \ a whenever a is a positive integer, the "divides" relation is reflexive. 


In some relations an element is related to a second element if and only if the second 
element is also related to the first element. The relation consisting of pairs (x yj where 
x and y are students at your school with at least one common class has (his property. 
Other relations have the property that if an element is related to a second element, then 
this second element is not related to the first. The relation consisting of the pairs (,r. yl 
where x and v arc students at your school where x lias a higher grade poult average than 
y has this property. 

DEFINITION 4. A relation R on a set A is called .symmetric if (b, a) E R when¬ 
ever (a, h) e R. for ai£ ,4. A relation R on a set A such that { a, b ) 6 R and 
{b, a) £ R only if a - b, for a, b E A , is called antisymmetric. 


That is, a relation is symmetric if and only if a is related to b implies that b is related 
to a. A relation is antisymmetric if and only if there arc no pairs of distinct elements 
a and h with a related to h and b related tn u. The terms symmetric and antisymmetric 
are not opposites, since a relation can have both of these properties or may lack both of 
them (see Exercise 6 at the end of this section), A relation cannot be both symmetric 
and antisymmetric if it contains some pair of the form [a. h) where a A b. 


Which of the relations from Example 7 are symmetric and which are antisymmetric? 

Solution: The relations Ari and R 3 are symmetric, because in each case lb. a) belongs 
to the relation whenever la, b) does. For Ri, the only thing to check is that both (2, I) 
and (1,2) are in the relation. For R\. it is necessary to check that both (1, 2) and (2,1) 
belong to the relation, and (1,4) and (4, 1) belong to the relation. The reader should 
verify that none of the other relations is symmetric. This is done by finding a pair (a, b) 
so that it is in the relation but (b, a) is not. 

R 4 . As, and % are all antisymmetric. For each of these relations there is no pair of 
elements u and b with a A b such that both (a. b) and (b, a) belong to the relation. The 
reader should verify that none of the other relations is antisymmetric. This is done by- 
finding a pair ( a , b) with a ^ h so that t a , b) and [b, a) are both in the relation. ■ 


Which of the relations from Example 5 are symmetric and which are antisymmetric? 

Solution: The relations R\. R A , and R t , are symmetric. A*;, is symmetric, for if a - b 
or a ~ -b, then h — a or b = -a. is symmetric since a = b implies that b - a. 
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K 6 is symmetric since a + b s 3 implies that b + a =£ 3. The reader should verify that 
none of the other relations is symmetric. 

The relations R] , /?;. R 4 , and are antisymmetric. R\ is antisymmetric because 
the inequalities a ^ b and b s a imply that a = b. Rj is antisymmetric since it is 
impossible for a > h and h > a. /t ’ 4 is antisymmetric, since two elements are related 
with respect to R* if and only if they are equal. R s is antisymmetric since it is impossible 
that a = b - 1 -1 and b = a + 1 . The reader should verify that none of the other relations 
is antisymmetric. ■ 

EXAMPLE 12 

Is the “divides" relation on the set of positive integers symmetric? Is it antisymmetric? 

Solution: This relation is not symmetric since 1 2, but 2 / 1. It is antisymmetric, for 
if a and b tire positive integers with a \ b and 1 !) j a, then a - b (the verification of this 
is left as an exercise fur the reader), ■ 

Let R be the relation consisting of all pairs (x, y) of students at your school where 
x has taken more credits than y. Suppose that x is related to v and y is related toThis 
means that x has taken more credits than y and y has taken more credits than z. We can 
conclude that a has taken more credits than 2 , so that x is related to 2 . What we have 
shown is that R has the transitive property, which is defined as follows. 

DEFINITION 5. A relation If on a set A is called tranmiw if whenever (a, b) G R 
and (b, c) 6 R, then (a. c ) G R, for a,b,cG A. 

EXAMPLE 1.1 

Which of the relations in Example 7 are transitive? 

Solution: A 4 , R$, and /?& are transitive. For each of these relations, we can show that it 
is transitive by verifying that if (a, b) and (b, e) belong to this relation, then (a, c) also 
does. For instance, R 4 is transitive, since (3, 2) and (2, 1), (4,2) and (2, 1), (4, 3) and 
(3, 1), and (4, 3) and (3, 2) are the only such sets of pairs, and (3, 1), (4, 1), and (4, 2) 
belong to R,\. The reader should verify that R 5 and R h are transitive. 

R- is not transitive since (3,4) and (4, 1) belong to R h but (3, I) does not Rn is 
not transitive since (2, 1) and (1. 2) belong to R 2 , but (2. 2) does not. Rt, is not transitive 
since (4, 1) and (1, 2) belong to R\. but (4, 2) does not, ■ 

EXAMPLE U 

Which of the relations in Example 5 arc transitive? 

Solution: The relations R u R z , Rt, and R 4 are transitive, R t is transitive since a £ b 
and b £ c imply that a s c. Rj is transitive since a > b and b > c imply that a > c. 

R-. is transitive since a = Lb and b = ±c imply that a = ±.c. R 4 is clearly transitive, 
as the reader should verify. R 5 is not transitive since ( 2 . 1 ) and (1,0) belong to % 
but (2, 0) does not. R t is not transitive since (2. 1) and (1, 2) belong to R„. but (2, 2) 
does nol. g 

EXAMPLE IS 

Is the "divides" relation on the set of positive integers transitive? 
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EXAMPLE 16 


EXAMPLE 17 


EXAMPLE 18 


Solution: Suppose that a divides b and b divides c. Then there are positive integers k 
and / such that b = ak anti f - hi. Hence, c = akl, so that a divides c. It follows that 
this relation is transitive. ■ 


The next example shows how to count the number of relations with a specified 
property. 


i low many reflexive relations are there on a set with n dements? 

Solution: A relation R on a set A is a subset of A X A, Consequently, a relation is 
determined by specifying whether each of the ir ordered pairs in A x A is in R. How¬ 
ever, if R is reflexive, each of the n ordered pairs (a, a) for a € A must be in R, Each 
of the other rtin - 1) ordered pairs of the form (a, b) where a X b may or may not 
be in R, Hence, by the product rule for counting, there are 2 n(H Jj reflexive relations 
[this is the number of ways to choose whether each element (a, b) with a ¥> b belongs 
to??]. ■ 


The number of symmetric relations and the number of anti symmetric relations on 
a set with n elements can be found using reasoning similar to that in Example 16 (see 
Exercise 25 at the end of this section). Counting the transiti ve relations on a set with n 
elements is a problem beyond the scope of this boot. 


COMBINING RELATIONS 


Since relations from A to B are subsets of A X 6 , two relations from A to B can be 
combined in any way two sets can be combined. Consider the following examples. 


Let A = {1, 2,3} and B = {1,2,3,4}. The relations /?[ = {(1, 1), (2,2), (3, 3)} and 
fa ~ {(I, 0,(1, 21(1,3), (1,4)} can be combined to obtain 

R) J fa = {(1, 0,0. 2), 1.1, 31(1,4), (2, 2), (3. 3)}, 

R] n fa = {(L 1)}, 

R\ ~ = {(2,2), (3, 3)f, 

= {0.2). (1, 3), (1,4)}. ■ 


Let A and B be the set of all students and the set of all courses at a school, respectively. 
Suppose that Rj consists of all ordered pairs fa, b), w'here a is a student who has taken 
course b. and R 2 . consists of all ordered pairs (a, b), where a is a student who requires 
course h to graduate. What are the relations R t U R 2 , fa H R 2 , Ri © fa, Ri ~ fa, and 
rv - R ’ 

Solution: The relation R } U R? consists of all ordered pairs (a, b). where a is a student 
who either has taken course b or needs course b to graduate, and Ri tl R 2 is the set 
of all ordered pairs (a b). where a is a student who has taken course h and needs this 
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EXAMPLE 19 


EXAMPLE 20 


course to graduate. Also, If i consists of all ordered pairs (a, b), where student a has 

taken course b but does not need it to graduate or needs course b to graduate but has not 
taken it. R\ - Rz is the set of ordered pairs (a, b), where a has taken course b but 
does not need it to graduate; that is, b is an elective course that a has taken. Rz - R i is 
the set of all ordered pairs (a, b) t where b is a course that a needs to graduate but has 
not taken. ® 


There is another way that relations are combined which is analogous to the com¬ 
position of functions. 



The following example illustrates how composites of relations are formed. 


What is the composite of the relations R and S where R is the relation from {1,2,3} 
to {1, % 3,4} with R = {(1, l), (1,4), (2,3), (3,1), (3,4)} and S is the relation from 
{1. Z 3,4} to {0.1,2} with S = {(1,0). (2,0), (3,1), (3,2). (4,1)}? 

Solution: S o R is constructed using all ordered pairs in R and ordered pairs in 5, where 
the second element of the ordered pair in R agrees with the first element of the ordered 
pair in S. For example, the ordered pairs (2,3) in R and (3,1) in S produce the ordered 
pair (2,1) in S o R. Computing all the ordered pairs in the composite, we find 

So R = {{1,0),(1,1),(2,1),(2,2),(3.0),(3, 1)}. ■ 


The powers of a relation R can be inductively defined from the definition of a com¬ 
posite of two relations. 

&Kft>tfTlQft7. UtafcciirlminiJftltoM * Tbeji ‘mC s = L.i 1 

ITT irfi’ant iifc&afirofy by 

I 1 mi m fT o * 


The definition shows that R 1 = R o R, R 3 = R 2 o R - (R o R) a R, and so on. 


Let R = {(1,1), (2,1), (3,2), (4,3)}. Find the powers SP, n = 2, % 4,. 

Solution: Since R l = R°R, we find that ff 2 = {{1,1), (2,1), (3,1),(4,2)}. Further¬ 
more, since R 3 = R 2 ° R, R 3 ~ {(1,1),(2,1), (3,1), (4,1)}. Additional computation 
shows that if 4 is the same as ft 3 , so R 4 = {{1,1), (2,1), (3,1), (4,1)}. It also follows that 
R n = R- for n = 5, 6, 7,.... The reader should verify this. ■ 
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The following theorem shows that the powers of a transitive relation are subsets of 
this relation. It will he used in Section 6.4. 

THEOREM 1 The relation R on a set A is transitive if and only if E* C R for n - 1,2,3, _ 

Proof: We first prove the if part of the theorem. Suppose that R h C R for « = 

1.2,3. In particular, R 1 C R. To see that this implies R is transitive, note that 

it (a, h ) £ R and [b, c) E R, then by the definition of composition, (a, r) £ R 1 . Since 
R 2 C R, this means that (a, c) E R. Hcncc R is transitive. 

We w ill use mathematical induction to prove the only if part of the theorem. Note 
that this part of the theorem is trivially true for n — L 

Assume that R n C R where n is a positive integer. This is the inductive hypothesis. 
To complete the inductive step we must show that this implies that R nJ ~ 1 is also a subset 
of R To show this, assume that (a, h) £ R n+] , Then, since R*' 1 = R ,! o R, there is an 
element x with x E A such that (a, x) E R and (x, b) E R n . The inductive hypothesis, 
namely, that R" C R, implies that (x,b) E R. Furthermore, since R is transitive, and 
hr t) E R and (a, b) G R it follows that (a, h) G R, This shows that R n 11 C R com¬ 
pleting the proof. n 


Exercises 


1. Lisl the ordered pairs in [he relation R from A = 
•|t), 1,2, 3.41 to R - <U 1,2, 3} where (a. ft) G R if and 
only 1 1 

a) a - h. h) u -T b = 4. 

cj a > h. d) u ! b. 

e) gedfo. />) I fi lemur b} = 2 

2. a) List all the ordered pairs in the relation R - ((a. b)\ 

a divides b) on the set {1. 2, 3,4. 5. ft}. 
b> Display this relation graphically, as was done in 
Example 4, 

e) Display this relation in tabular form, as was done in 
Example 4. 

3. For each of the following relations on the set {1.2.3,4}. 
decide whether it is reflexive, whether it is symmet¬ 
ric. whether it is antisymmetric, and whether it is 
transitive, 

a) {(2. 2;. (2. 31 (2.4). (3. 2). (3.3i. (3.4)} 

b) {(]. I), i 1. 2t, (2, 1 m 2. A. i.3.3u 4.4)} 

c) {(2,4). 1 4. 2)} 

d) {(I. 21. (2,3). (3,4)| 

e) {i.l. il i„2. 2).! r 3). (4, 4)f 

fi It I. 3). (t, 4). {2, 3). (2. 4), (3. 1), {3,4)} 

4. Determine whether she relation R on the set of all peo¬ 
ple is reflexive, symmetric, antisymmetric, and/or tran¬ 
sitive, where (/<, ft) E R if and only if 

a) a is ruller than b. 

b) a and h were (tom on the same day, 
cl a has the same lirst name as {>, 

d) a and h have a common grandparent. 


5. Determine whether the relation R on the set of all in¬ 
tegers is reflexive, symmetric, antisymmetric, and/or 
transitive, where (.e, v) E R if and only if 

a) x A y. b) xy - I 

c) x - y - I or x = y 1. d) Jt- v (mod 7), 

e) x is a multiple of v. 

fl x and y are both negative or both nonnegative, 
g) < - v : , h) x >™. 

6. Give an example of a relation on a set that is 

a) symmetric and antisymmetric, 

b) neither symmetric nor antisymmetric, 

cr A relation R on the set A is irreflexive if for every a € 
A, (a. a) E R. That is. R is irreflexive if no element in A is 
related to itself. 

7. Which relations in Exercise 3 are irreflexive? 

8. Which relations in Exercise 4 are irreflexive? 

‘T Can a relation on a set be neither reflexive nor irre¬ 
flexive? 

A relation R is called asymmetric if (a, h) E R implies that 

(h, a) £ R, 

10 . Which relations in Exercise 3 are asymmetric? 

11. Which relations in Exercise 4 arc asymmetric? 

12. Must an asymmetric relation also he antisymmetric? 
Must an antisymmetric relation be asymmetric? Give 
reasons for your answers. 


PDF created with pdfFactory trial version www.pdffactorv.com 








6.1 Exercises 38? 


1.1, How many different relations arc there from a .set with 
m elements to a set with n elements? 

■:./ Let ft be a relation from a set ,4 to a set B. The inverse 
relation from H to A, denoted by R"\ is the set of ordered 
pairs {(£, ti) \ {a, if) t= R'l The complementary relation R 

is the set of ordered pairs {(«, b) j (fj, b) £ R\, 

14. 1 ,et R be the relation R = {(cr, b) j a < h\ on the set of 
integers Find 

a) R 

hi R. 

15. I ,cr R be the relation R i(a r b) \ a divides 6} on the 
set of positive integers. Find 

a) R 

b) R 

16. Let R be the relation on the set of all states in the United 
States consisting of pairs (a.b'i where state a borders 
state h. Find 

a) R . 
h) R 

17. Suppose that the function/ from A. to B is a one-to-one 
correspondence. I.ct R be the relation that equals the 
graph of/ That is, R - {to, /(«)) | a e A}. What is 
the inverse relation R ! ? 

18. LetR; - 1(1.21,6:. 31,(3,41} and R 2 = {£1, 4(1,2), 
(2.11. (2.2). (2.31. 1 3,1), (3,2), (3.3), (.1, 4)} be relations 
front {1,2.3Ho '1,2, 3.4}, Find 

a) R: L R- 

b) A', n fit. 

c) !■: ■ R 

d) R; R,. 

19. I ,ct A be the set of students at your school and R the set 
of books in the school library, l.ct R\ and of be the re¬ 
lations consisting of all ordered pairs In, b) where stu¬ 
dent a is required to read book b in a course, and where 
student « has read book h, respectively. Describe the 
ordered pairs in each of the following relations. 

a) R] J R : 
hi R r R 

c) R; 2 R'_ 

d) R\ - R : 

e) R : - R, 

20. I cl A* be (he relation {(1, 2). (1,3). (2, 3), (2 4), (3. 1)}, 
and let A be the relation {(2, 1), (3, 1), (3, 2). (4, 2)}. Find 
S a R 

21. Let R be the relation oil the set of people consisting of 
pairs (a. hi I where a is a parent of b. Let S be the relation 
on the set of people consisting of pairs jfi. h) where a 
and b are siblings (brothers or sisters). What arc S o R 
and R a s'! 

22. List the 16 different relations on the set {0, I}. 

23. How- many of the 16 different relations on {(), 1} contain 
the pair (0. I 

24. Which of the 16 relations on }0. 1}, which you listed in 
Exercise 22, are 


a) reflexive? 

b) i [reflexive 7 
C) symmetric? 

d) antisymmetric? 

c) asymmetric? 
f) transitive? 

*25. How many relations are there on a sel with n elements, 
that are 

a) symmetric? 

b) antisymmetric? 

c) asymmetric? 

d) irreflexive? 

e) reflexive and symmetric'’ 

f) neither reflexive nor irreflexivc? 

*26. How many transitive relations arc there on a set with n 
elements if 

a) u = I? h) n = 2? c) n = 3.’ 

27. Find the error in the "proof" of the following “theo¬ 
rem." 

“Theorem": Let R be a relation on a set .4 that is sym¬ 
metric and transitive. Then A is reflexive, 

“Proof": Let a 6 A. Take an element b £ A such that 
(a. b} i: R Since R is .symmetric, we also have (b, a) E 
R. Now using the transitive property, we can conclude 
that (ci. n) G R since (ft. b) £ R and (h : a) E R. 

28. Suppose that R and 2 are reflexive relations on a set A. 
Prove or disprove each of the following statements. 

a) R U S is reflexive, 
hi R H .y is reflexive. 

c) R & S is irreflexivc. 

d) R - S is irreflexivc. 

e) S o R is reflexive. 

29. Show that the relation R on a set A is symmetric if and 
only if R ■ A 1 1 where R~ is the inverse relation. 

30. Show that the relation R on a set A is antisymmetric if 
and only if R n R 1 is a subset of the diagonal relation 
A ■ {(i/.ui | u E .4). 

31. Show that the relation R on a set A is reflexive if and 
only if the inverse relation R ' is reflexive. 

32. Show that (he relation R on a sel A is reflexive if and 
only if the complementary relation R is irreflexive. 

33. I .et R be a relation that is reflexive and transitive. Prove 
lhal R” = R for all positive integers n. 

34. Let R be the relaLion on the set {1,2, 3, 4. 5} containing 
the ordered pairs (1, 1), (1,2), (1,3), (2, 3), (2. 4), (3, 1), 
(3,4), (3, 5), (4, 2), (4.5), (5.1), (5, 2), and (3.4). Find 
a) R . b) R\ c) R 4 d) A' - 

35. Let R be a reflexive relation on a set A. Show that R" is 
reflexive for till positive integers n. 

*36. Let R be a symmetric relation. Show that R” is sym¬ 
metric for all positive integers n. 

37. Suppose that the relation R is irreflexivc. Is R ] neces¬ 
sarily irreflexivc 7 Give a reason for your answer, 
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6.2 _ 

//-ary Relations and Their Applications 

INTRODUCTION 


Relationships among elements of more than two sets often arise. For instance, there is a 
relationship involving the name of a student, the student’s major, and the student’s grade 
point average. Similarly, there is a relationship involving the airline, flight number, 
starting point, destination, departure time, and arrival time of a flight. An example of 
such a relationship in mathematics involves three integers where the first integer is 
larger than the second integer, which is larger than the third. Another example is (he 
betweenness relationship involving points on a line, such that three points are related 
when the second point is between the first and the third. 

We will study relationships among elements train more than two sets in this section. 
These relationships are called n-arv relations. These relations are used lo represent 
Computer databases. These representations help us answer queries aboul the informa¬ 
tion stored in databases, such as: Which flights land at O'Hare Airport between 3 a m. 
and 4 a m 7 Which students at your school are sophomores majoring in mathematics or 
computer science and have greater than a 3.0 average? Which employees of a company 
have worked tor the company less than 5 years and make more than $50,000? 

/i-ARY RELATIONS 


We begin with a definition. 

DEFINITION 1. Let A\,A 2 ,.... A n besets. An n-ary relation out&esesets is a 
subset of At x A 2 X • • • x A„. The sets A\, Aj, ,.., A* are called the domaitu of Eis 
relation, and n is called its degree. 


Let R be the relation consisting ot triples (a. b, r) w here a. h, and c are integers with 
a < b < c. Then (1,2, 3) 6 R, but (2,4, 3) £ R The degree of this relation is 3. its 
domains are ail equal to (he set of integers. ■ 


Let R be the relation consisting of 5-tuples (A, A. S, D, T) representing airplane flights, 
where A is the airline, A r is the flight number, $ is the storting point, D is the destination, 
and T is the departure time. For instance, if Nadir Express Airlines has flight 963 from 
Newark to Bangor at 15:00, then (Nadir, 963. Newark, Bangor, 15:00) belongs to R. 
The degree of this relation is 5, and its domains are the set of all airlines, the set of 
flight numbers, the set of cities, the set of cities (again), and the set of times. ■ 


DATABASES AND RELATIONS 


The time required to manipulate information in a database depends on how this in¬ 
formation is stored. The operations of adding and deleting records, updating records. 
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EXAMPLE 3 


searching for records, and combining records from overlapping databases are performed 
millions of times each day in a large database. Because of the importance of these 
operations, various methods for representing databases have been developed. We will 
discuss one of these methods, called the relational data model, based on the concept 
of a relation, 

A database consists of records, which are «-tuples, made up of fields. The fields 
are the entries of the n-tuples. For instance, a database of student records may be made 
up of fields containing the name, student number, major, and grade point average of the 
student. The relational data model represents a database of records as an n-ary relation. 
Thus, student records are represented as 4-tuples of the form (STUDENT NAME. ID 
NUMBER, MAJOR, CPA). A sample database of six such records is: 

(Ackermann, 231455, Computer Science, 3.88) 

(Adams, 888323, Physics, 3.45) 

(Chou, 102147, Computer Science, 3.79) 

(Goodfriend. 453876, Mathematics, 3.45) 

(Rao, 678543, Mathematics, 3.90) 

(Stevens, 786576, Psychology, 2.99), 

Relations used to represent databases are also called tables, since these relations are 
often displayed as tables. For instance, the same database of students is displayed in 
Table 1. 

A domain of an n-ary relation is called a primary key when the value of the n-tuple 
from this domain determines the n-tuple. Thai is, a domain is a primary key when no 
two 'Huples in the relation have the same value from this domain. 

Records arc often added to or deleted from databases. Because of this, the property 
that a domain is a primary key is time-dependent. Consequently, a primary key should 
be chosen that remains one whenever the database is changed. This can be done by 
using a primary' key of the intension of the database, which contains all the n-tuples 
that can ever be included in an n-ary relation representing this database. 

Which domains are primary keys for the /i-ary relation displayed in Table 1, assuming 
that no n-tuples will be added in the future? 


TABLE 1 

Student Name 

ID Number 

Major 

GFA 

Ackerman 

231455 

Computer Science 

3.88 

Adams 

888323 

Physics 

3.45 

Chou 

102147 

Computer Science 

3.79 

Goodfriend 

453876 

Mathematics 

3.45 

Rao 

678543 

Mathematics 

3.90 

Stevens 

786576 

_ 

Psychology 

2.99 
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EXAMPLE 4 


EXAMPLE 5 


Solution; Since there is only one 4-tuple in this table for each student name, the domain 
of student names is a primary key. Similarly, the ID numbers in this table are unique, 
so that the dom ai n of ID numbers is also a primary'' key. However, the domain of major 
fields of study is not a primary key, since more than one 4-tuple contains the same major 
field of study. The domain of grade point averages is also not a primary key, since there 
are two 4-tuples containing the same GPA (which ones?). ■ 


Combinations of domains can also uniquely identify n-tuples in an n-ary relation. 
When the values of a set of domains determines an n-tuple in a relation, Ihe Cartesian 
product of these domains is called a composite key. 


Is the Cartesian product of the domain of major fields of study and the domain of CPAs 
a composite key for ihe n-ary relation from Table 1, assuming that no n-tuples are ever 
added? 

Solution: Since no two 4-tuples from this table have both the same major and the same 
GPA, this Cartesian product is a composite key. ■ 


Since primary and composite keys tire used to identify records uniquely in a 
database, it is important that keys remain valid when new records are added to the 
database. Hence, checks should be made to ensure that every new record has values 
that are different in the appropriate field, or fields, from all other records in this table. 
For instance, it makes sense to use the student identification number as a key for stu¬ 
dent records if no two students ever have the same student identification number. A 
university should not use the name field as a key, since two students may have the same 
name (such as John Smith). 

There are a variety of operations on n-ary relations that can be used to form new 
n-ary relations. Two of these operations will be discussed here, namely, the projection 
and join operations. Projections are used to form new n-ary relations by deleting the 
same fields in every record of the relation. 


DEFINITION 2. The projection maps the n-tuple (oj, ..,, a n ) to ihe 

m-tuple (a;,, a, v .. where m < n. 


In other words, the projection P (ui ^ tim deletes n- m of the components of an n-tuple. 
leaving the ijth, rith,..., and ;,„tb components. 


What results when the projection P u is applied to the 4-tuples (2, 3, (1,4), (Jane Doe, 
23411 KX)1, Geography. 3.14), and {a*. at, a-), Oi) r ’ 

Solution: The projection F u sends these 4-tuples to (2,0), (Jane Doe, Geography), 
and (a], tfj), respectively. ■ 


The following example illustrates how new relations are produced using projections. 
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EXAMPLE 6 What relation results when the projection f', 4 is applied to the relation in Table 1 ? 

Solution; When the projection P \.4 is used, the second and third columns of the table 
are deleted, and pairs representing student names and grade point averages are obtained. 
Table 2 displays the results of this projection, ■ 


Fewer rows may result when a projection is applied to the table for a relation. This 
happens when some of the /t-tuples in the relation have identical values in each of 
the m components of the projection, and only disagree in components deleted by the 
projection. For instance, consider the following example. 


EXAMPLE 7 What is the table obtained when the projection P { 1 is applied to the relation in Ta¬ 

ble 3? 

Solution: Table 4 displays the relation obtained when P 1.2 is applied to Table 3. Note 
that there tire fewer rows after this projection is applied. ■ 


The join operation is used to combine two tables into one when these tables share 
some identical lie Ids. For instance, a table containing lields for airline, flight number, 
and gate, and another table containing fields for flight number, gate, and departure time 
can be combined into a table containing fields for airline, flight number, gate, and de¬ 
parture time. 


DEFINITION 3. Let R be a relation of degree tit md S.a relation of degree n. 
The/ifert J P (R, S’), where p < m and p s n, is a relation of degree m + n - p that 
consists of all (m + ii - p)-tuples (a h a % ,,,a m - p ,cuC 2 >...,c p ,b h b%.,., b a - p ), 
where the ro tuple (fli, 02 . *.•,««- P , ct, C 2 ,, c p ) belongs to If and the u-tuple 
to, c' 2 ,.... c p , hi, bz ..., b„- P ) belongs to S. 


TABLE 3 

Student 

Major 

Course 

G1 an set 

Biology 

bi m 

Glattser 

Biology 

MS 475 

Giauser 

Biology 

FY 4|() 

Marcus 

Mathematics 

MS 511 

Marcus 

Mathematics 

MS 603 

Marcus 

Mathematic? 

CS 522 

Miller 

Computer Science 

MS 575 

Miller 

Computer Science 

CS 455 


TABLE 2 

Student Name 

GPA 

Ackerman 

3.88 

Adams 

5,45 

Chou 

3.79 

Goodfriend 

3.45 

Raw 

3.90 

Stevens 

2.99 


TABEE 4 

Student 

Major 

Giauser 

Marcus 

Miller 

Biology 
Mathematics 
Computer Science 
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TABLEf 


TABLE (< 

Professor 

Department 

Course 

Number 

Department 

Course 

Number 

Room 

Time 

Cm?. 

Zoology 

335 


Computer Sc re nee 

518 

X521 

2:00 pm. 

Cm? 

Zoology 

412 


Mathematics 

575 

N502 

3:00 p.m. 

Karber 

Psychology 

501 


Mathematics 

611 

N52I 

4:00 p.m. 

Faroe r 

Psychology 

617 


Physics 

544 

B505 

4:00 p.m. 

Gram mer 

Physics 

544 


Psychology 

501 

A100 

3:00 pm. 

Gtammer 

Physics 

551 


Psychology 

ft 17 

A110 

U:00a.m 

Rosen 

Computer Science 

518 


Zoology 

335 

A100 

9:00 a.m. 

Rosen 

Mathematics 

575 


Zoology 

412 

A100 

8:00 am 


In other words, the join operator J r produces a new relation front two relations by 
combining all m-luplcs of the first relation with all zt-tuples of the second relation, 
where the last p components of the m -tuples agree with the first p components of the 
H-tuples. 


EXAMPLE S What relation results when the join operator 7; is used to combine the relation displayed 

in Tables 5 and 6? 

Solution: The join J 2 produces the relation shown in Table 7. ■ 


There are other operators besides projections and joins that produce new relations 
from existing relations. A description of these operations may be found in books on 
database theory. 


TABLE 7 

Professor 

Department 

Course Number 

Room 

Time 

Cruz 

Zoology 

535 

A100 

9:00 a.m 

Cruz 

Zoology 

412 

A100 

8:00 am. 

Parher 

Psychology 

501 

A100 

3:00 p.m. 

Farber 

Psychology 

617 

A110 

1 1:00 a.m. 

Grant mer 

Physics 

544 

B505 

4:00 p.m. 

Rosen 

Computer Science 

518 

N521 

2:00 p.m 

Rosen 

Mathematics 

57 5 

N5.02 

3:00 pm 
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TABLES 

Airline 

Flight Number 

Gate 

Destination 

Departure Time 

Nadir 

122 

34 

Detroit 

08:10 

Acme 

221 

22 

Denver 

08:17 

Acme 

122 

33 

Anchorage 

08:22 

Acme 

323 

34 

Honolulu 

08:30 

Nadir 

199 

13 

Detroit 

08:47 

Acme 

222 

22 

Denver 

09:10 

Nadir 

322 

34 

Detroit 

09:44 


Exercises 

1. List the triples in the relation {{a, h, c) j a, h, and r are 
integers with 0 < a < h < c < 5, 

2. Which 4-luples are in the relation {(a, b, c. d) | a, h, c. 
and d are positive integers with abed = 6}? 

3. l.ist the 5-tuples in the relation in Table 8. 

4. Assuming that no new n-tuples are added, find all the 
primary keys for the relations displayed in 

a) Table 1 

b) Table 3. 

c) Table 6. 

d) fable 8 . 

5. Assuming that no new' n-tuples are added, find a com¬ 
posite key with two fields containing the Airline fieJd 
for the database in Table 8- 


6 . What do you obtain when you apply the projection 
P:,\* to the 5-tuple (a, b, c, d, e)l 

7. Which projection mapping is used to delete the first, 
second, and fourth components of a 6 -tuplel' 

8 . Display the table produced by applying the projection 
P UA to Table 8 . 

9. Display the table produced by applying the projection 
P L 4 to Table 8 . 

10 . How many components are there in the «-tuples in 
the table obtained by applying the join operator ,'i 
to two tables with 5-tuples and 8 -tuples, respect¬ 
ively? 

11 . Construct the table obtained by applying the join oper¬ 
ator d: to the relations in Tables 9 and 10. 


table; 9 

Supplier 

Pari Number 

Project 

23 

1092 

! 

23 

1101 

3 

23 

9048 

4 

31 ! 

4975 

3 

31 

3477 

2 

32 ; 

6984 

4 

32 i 

9191 

2 

L2i.... , 

1001 

i 


TABLE ID 

Part 



Color 

Number 

Project 

Quantity 

Code 

1001 


14 

$ 

i 092 

• 

T 

2 

1101 

3 

l 

1 

3477 

2 

25 

2 

4975 

3 


2 

6984 

4 

10 

i 

9048 

4 

12 

2 

9191 

? 

80 

4 
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6.3 _ 

Representing Relations 

INTRODUCTION 


There are many ways to represent a relation between finite sets. As we have seen, one 
way is to list its ordered pairs. In this section we will discuss two alternative methods 
tor representing relations, One method uses zero-one matrices. The other method uses 
directed graphs. 


REPRESENTING RELATIONS USING MATRICES 

A relation between finite sets can be represented using a zero-one matrix. Suppose that 

/? is a relation from A » {cp. a 2 . a m \to R = {b\, b 2 ,. , b a ). (Here the elements of 

the sets ,4 and B have been listed in a particular, hut arbitrary, order. Furthermore, when 
A = B we use the same ordering for 4 and B.) The relation R can be represented by 
the matrix M s = where 

fl if (tij, bj) G R, 

‘ [0 if (ui,b])£R. 

In other words, the zero-one matrix representing R has a i as its ( 1 , j) entry when a, 
is related to bj, and a 0 in this position it «, is not related to bj. (Such a representation 
depends on the orderings used far4 and B.) 

The use of matrices to represent relations is illustrated in the following examples. 


EXAMPLE 1 


Suppose that 4 - (1,2, 3} and B = {1. 2}. Let R be the relation from 4 to B containing 
(a, b) if a E 4, ft e B, and a > b. What is the matrix, representing Rifaj = I,« 2 - 2, 
and as = 3, and h\ - 1 and h 2 = 2? 


Solution: Since R = {(2, 1), (3, 1), (3. 2)1, the matrix for R is 


M* 


0 0 ] 


The Is in M fi show that the pairs (2,1). (3, I ), and (3, 2) belong to R. The Os show that 
no other pairs belong to R. g 


EXAMPLE 1 


LetA - fiij, o;, od and B - {£>,, b 2 . b$, i> 4 . b^\, Which ordered pairs are in the relation 
R represented by the matrix 


M* 


Jo 10 0 0] 

1 0 i i 0! ? 
1 0 1 0 i J 
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Solution: Since R consists of those ordered pairs to, bf) with m., - ], it follows that 

R - (to. fell, to, b[). (u 2 , h), to. b A % to. h), to. b 3), to, £5}}. ■ 


1 


1 


] 


FIGURE l 
The Zero-One 
Matrix for a 
Reflexive 
Relation. 


The matrix of a relation on a sol, which is a square matrix, can be used to determine 
whether the relation has certain properties. Recall that a relation R on ,4 is reflexive if 
(et, a) £ R whenever a 6 A. Thus, R is reflexive if and only if to, to E R for i = 

1. 2. n. Hence, R is reflexive if and only if m„ - 1, for i = 1,2, —, n. In other 

words, A‘ is reflexive if all the elements on the main diagonal of Mr are equal to 1, as 
shown in Figure 1, 

Tire relation R is symmetric if (a. b) G R implies that (b, a) G R. Consequently, the 
relation R on the set A = to ,a 2 .is symmetric if and only if (a j, to E R when¬ 

ever (a f . ttj) E R In terms of the entries of Mr, R is symmetric if and only if m j, = 1 
whenever m/y = i. This also means my,- = 0 whenever m l} = 0. Consequently, R is 

symmetric if and only if m, ; ~ m ,,, for all pairs of integers i and / w ith 1 - 1,2. n 

and j - 1,2. n. Recalling the definition of the transpose of a matrix from Section 

2,6, we see that R is symmetric if and only if 

Mr - (Mr)', 

that is, if Mr is a symmetric matrix. The form of the matrix for a symmetric relation is 
illustrated in Figure 2(a). 

The relation R is antisymmetric if and only if (a, b) e R and (/?, a) £ R imply that 
a = b. Consequently, the matrix of tin antisymmetric relation has the property that 
if m,y - 1 with / A /, then mj, = 0. Or, in other words, either m l f = 0 or ir. j: - 0 
when i ^ j. The form of the matrix for an antisymmetric relation is illustrated in 
Figure 2(b). 


EXAMPLE i 


Suppose that the relation R on a set is represented by the matrix 


f- ; 


l 

o' 

i 

N< 0 

Mr - 

1 

■' X 


0 

1 


n ‘ I 


Is R reflexive, symmetric, and/or antisymmetric? 


M Svm 


llllCUh 



i.bi -Viitthymmotiii. 

FIGURE 2 
The Zero-One 
Matrices for 
.Symmetric and 
Antisymmetric 
Relations. 


Solution: Since all the diagonal elements of this matrix are equal to 1, R is reflexive. 
Moreover, since Mr is symmetric, it follows that R is symmetric. It is also easy to see 
that R is not antisymmetric, ■ 


The Boolean operations join and meet (discussed in Section 2.6) can be used to 
find the matrices representing the union and the intersection of two relations. Suppose 
that R] and Ri are relations on a set A represented by the matrices M* and Mr 2 , 
respectively. The matrix representing the union of these relations has a 1 in the positions 
where either Mr. or Mr, has a 1. The matrix representing the intersection of these 
relations has a I in the positions where both Mr, and Mr, have a 1. Thus, the matrices 
representing Ihc union and intersection of these relations are 

Mji.i'ft - Mr, vM|t and Mr. n * 3 = Mr, A Mr,. 
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EXAM PI i: 4 


EXAMPLE 5 


Suppose that the relations R\ and AS on a set A are represented by the matrices 



L 0 f 

[1 0 !' 

M fi - 

1 0 o 

and Mr, ~ j 0 1 1 


0 1 0 

1 0 0 


What are the matrices representing R i U AS and R{ fl ifj? 


Solution: The matrices of these relations are 


Ms 


R; ■= Mfl, yMj?, - 


1 0 1 


110 


[1 0 T 

=■ M,?! AMj, = 0 0 0 

[o 0 0 


We now rum our attention to determining the matrix for the composite of relations, 
This matrix can be found using the Boolean product of the matrices (discussed in Sec¬ 
tion 2,6) for these relations. In particular, suppose that A* is a relation from A to B and 
5 is a relation from B to C. Suppose that A, B, and C have m, n, and p elements, re¬ 
spectively. Let the zero-one matrices for S o R, fi, and S be M. ?oK = ^ [r i} ], 

and = Jr,,], respectively (these matrices have sizes rn X p,mX n, and n X p, re¬ 
spectively). The ordered pair ftp, c/j belongs to S o R if and only if there is an element 
b k such that {a;, bj.) belongs to R and (h k . cf) belongs to .S’. It follows that t, f ~ 1 if 
and only if r, k - s tj = 1 for some k. From the definition of the Boolean product, this 
means that 

Mjfo* = Ms O Ms. 


Find the matrix representing the relations So R where the matrices representing R and 
.S’ are 


fi o r 



0 1 O' 

- 1 10 

and 

M 5 - 

0 0 1 

jO 0 0 



1 0 1 


Solution; The matrix for .S’ o R is 
r l I 

M\ s Ms 0 Ms = ; 0 1 

[o 0 


n 

i 

Oj 


The matrix representing the composite of two relations can be used to find the 
matrix for M«». In particular, 

M** - Mfll 

from the definition of Boolean powers. Exercise 19 at the end of this section asks for a 
proof of this formula. 
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EXAMPLE 6 


EXAMPLE 7 



FIGURE 3 A 
Directed Graph. 


Find the matrix representing the relation F 2 where the matrix representing H is 

ft) 1 (U 
M„ = jO 1 I,. 

[1 0 oj 


Solution: The matrix fur R 2 is 


M = M 


:;i 

R 


r 0 1 I' 

1 1 1 

0 1 0 


REPRESENTING RELATIONS USING DIGRAPHS 


We have shown that a relation can be represented by listing all of its ordered pairs or by 
using a zero-one matrix. There is another important way of representing a relation using 
a pictorial representation. Each element of the set is represented by a point, and each 
ordered pair is represented using an arc with its direction indicated by an arrow. We 
use such pictorial representations when we think of relations on a finite set as directed 
graphs, or digraphs. 


MUNITION 1, A ttirecmdgmpk, mSgmphmmk (fc#9 set V of vertices (or 
nodes) tegatber wftfea set -B 0# ordered pairs of efeawattHf' ¥ salkgt (ex sm). 

The vests* a is called die initial vertex of the edge (*A),uied d>c xcitex b is called 
the terminal vertex of this edge. : ; Ki/ 


An edge of the form (a, a) is represented using an arc from the vertex a back to 
itself Such an edge is called a loop. 


The directed graph with vertices a, b, r, and d, and edges (a, b), >,a. ii 1 . (b, b ), (h, d), 
tc, a), fr, h), and (d, b ) is displayed in Figure 3. | 


The relation R on a set A is represented by the directed graph that has the elements 
of A as its vertices and the ordered pairs (a, b), where (a, b } E R, as edges. This as¬ 
signment sets up a one-to-one correspondence between the relations on a set A and the 
directed graphs with A as their set of vertices. Thus, every statement about relations 
corresponds to a statement about directed graphs, and vice versa. Directed graphs give a 
visual display of information about relations. As such, they are often used to study rela¬ 
tions and their properties. (Note that relations from a set A to a set B can be represented 
by a directed graph where there is a vertex for each dement of A ami a vertex 
for eadi element of B as shown in Section 6.1. However, when A = B, such 
representation provides much less insight than the digraph representations described 
here.} The use of directed graphs to represent relations is illustrated in the following 
examples. 
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EXAMPLE 8 


EXAMPLE 9 


EXAMPLE II) 


The directed graph of the relation 

R -- {(1, L), (i. 3). (2,1), (2,3), (2, 4), (3, 1), (3, 2), (4,1)} 
on the set {1,2, 3.4} is shown in Figure 4. ■ 


What are the ordered pairs in the relation R represented by the directed graph shown in 
Figure 5? 

Solution: The ordered pairs (x, y) in the relation are 

R = {(1.3), Cl. 4).(2.1).(2,2), (2. 3), (3, 11, (3, 3), (4.1), (4,3)}. 

Each of these pairs corresponds to an edge of the directed graph, with (2,2) and (3,3) 
corresponding to loops, ■ 


The directed graph representing a relation can be used to determine whether the 
relation has various properties. For instance, a relation is reflexive if and only if there is 
a loop at every vertex of the directed graph, so that every ordered pair of the form (a, ,c) 
occurs in the relation, A relation is symmetric if and only if for every edge between 
distinct vertices in its digraph there is an edge in the opposite direction, so that ( y, x) is 
in the relation whenever U, vJ is in the relation. Similarly, a relation is antisymmetric 
if and only if there are never two edges in opposite directions between distinct ver¬ 
tices. Finally, a relation is transitive if and only if whenever there is an edge from 
a vertex a to a vertex v and an edge from a vertex y to a vertex z, there is an edge 
from x to z (completing a triangle where each side is a directed edge with the correct 
direction). 


Determine whether the relations for the directed graphs shown in Figure 6 tire reflexive, 
symmetric, antisymmetric, and/or transitive. 

Solution: Since there me loops at every vertex of the directed graph of R . it is reflexive. 
R is neither symmetric nor antisymmetric since there is an edge from a to b but not one 
from b to u, but there are edges in both directions connecting b and c. Finally, R is not 
transitive since there is an edge from a to b and an edge from b to r, but no edge from 
a to r. 



of the Relation R. 



FIGURE S The Directed Graph 
of the Relation K„ 
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Cal Directed iiraph of R (b) Directed graph of S 

FIGURE 6 rhe Directed Graphs of the Relations E ans S, 


Since loops are not present at all the vertices of the directed graph of S, this rela¬ 
tion is not reflexive. It is symmetric and not antisymmetric, since every edge between 
distinct vertices is accompanied by an edge in the opposite direction. It is also not hard 
to see from the directed graph that S is not transitive, since (c, a) and (a, b) belong to S. 
bul (c, b) does not belong to S. ■ 


Exercises 


1. Represent each of the following relations on {1, 2, 3} 
with a matrix (with the elements of this set listed in 
increasing order). 

a) {(I. iM 1,2).(t, 3)} 

b) Ut2.2U3.3l} 

c) 1(1. 1).U.2HUH2,2U.X 3), (3.3)} 

(15 it l, 3 i. a ni 

2, List (he ordered pairs in the relations on {1, 2.3} corre¬ 
sponding to the following matrices (where the rows and 
columns correspond to the integers listed in increasing 
order). 


"1 0 3] 


"0 

01 

ri i n 

0 ! 0 ' 

hj 

0 

»l 

c) 1 1 (.) 1 ! 

i 0 ti 


0 

Oj 

.1 1 


3. How cart the matrix for a relation be used to determine 
whether the relation is irreflextve? 

4. Determine whether the relations represented by the 
matrices in Exercise 2 are reflexive, itreflexive, sym¬ 
metric, antisymmetric, and/or transitive. 

5. How can [he matrix for R. the complement of the rela¬ 
tion A\ be Sound from the matrix representing R, when 
fi is a relation on a finite set .4? 

h>. How can the matrix for R 1 , the inverse of the relation 
A', he found from the matrix representing A’, when R is 
a relation on a finite set A? 


7. Let R be the relation represented by the matrix 




oir 

i i o 

d o 1. 


Find the matrix representing 
a) A* b) R. c) R 2 
8. Let A| and ft be relations on a set A represented by the 
matrices 



'0 1 O' 


M,, - 

1 1 1 

1 0 0. 

and Ms, = 


Find the matrices that represent 

a) A, Uft. b) ft n ft. cl ft oft. 

d) ft oft. e) A, ® ft. 

9. Let A’ be the relation represented by the matrix 

0 I O' 

M/t =001. 

.1 I 0. 

Find the matrices that represent 
a) ft. b) ft. c) ft*. 

10. Draw the directed graphs representing each of the re¬ 
lations from Exercise l. 
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11. Draw the directed graphs representing each of the re¬ 
lations from Exercise 2. 


15. it 



b 

.A 


12. Draw the directed graph that represents the relation 
{(a. a), (a, b), (b, r). ( c, h), (c, d ), id, a), id, b}). 


In Exercises 13-15 list the ordered pairs in the relations rep- 




resented by the directed graphs. 



16. How can the directed graph of a relation R on a finite 
set A he used to determine if a relation is irrefkxive? 


17. Determine whether the relations represented by the di¬ 
rected graphs shown in Exercises 13-15 are reflex¬ 
ive, trreflexive. symmetric, antisymmetric, and/or tran¬ 
sitive. 


14. a b 




18. Given the directed graphs representing two relations, 
how can the directed graph of the union, intersection, 
symmetric difference, difference, and composition of 
these relations he found 1 ’ 



19. Show that if M* is the matrix representing the relation 
R, then is the matrix representing the relation R". 


6.4 


Closures of Relations 


INTRODUCTION 


A computer network has data centers in Boston. Chicago, Denver, Detroit, New York, 
and San Diego. There are direct, one-way telephone lines from Boston to Chicago, 
from Boston to Detroit, from Chicago to Detroit, from Detroit to Denver, and from New 
York to San Diego. Let R be the relation containing (a, b) if there is a telephone line 
from the data center in a to that in b. How can we determine if there is some (possibly 
indirect) link composed of one or more telephone lines from one center to another? 
Since not all links are direct, such as the link from Boston to Denver that goes through 
Detroit, R cannot be used directly to answer this. In the language of relations, R is 
noi transitive, so it does not contain all the pairs that can be linked. As we will show 
in tliis section, we can find all pairs of data centers that have a link by constructing 
the smallest transitive relation that contains R This relation is called the transitive 
closure of R. 

In general, let R be a relation on a set A. R may or may not have some property 
P, such as reflexivity, symmetry, or transitivity. If there is a relation S with property 
P containing R such that S is a subset of every relation with property P containing R. 
then S is called the closure of R with respect to P. (Note that the closure of a relation 
with respect to a property may not exist; see Exercises 15 and 35 at the end of this 
section. ) We will show how reflexive, symmetric, and transitive closures of relations 
can be found. 

CLOSURES 


The relation R =■ {(1, l), (1,2), (2,1), (3,2)} on the set ,4 = {1,2, 3} is not reflexive. 
How can we produce a reflexive relation containing R that is as small as possible? This 
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EXAMPLE 2 


can be done by adding (2,2) and (3,3) to R, since these are the only pairs of the form 
{a, a) that are not in R. Clearly, this new relation contains R. Furthermore, anv reflexive 
relation that contains R must also contain (2,2) and (3, 3). Because this relation contains 
R, is reflexive, and is contained within every reflexive relation that contains R. it is 
called the reflexive closure of R. 

As this example illustrates, given a relation R on a set A, the reflexive closure of 
R can he formed by adding to R all pairs of the form (a, a) with a £ A, not already 
in R. The addition of these pairs produces a new relation that is reflexive, contains R, 
and is contained within any reflexive relation containing R. We see that the reflexive 
closure of R equals R U A, where A = {■;/;, a) | a £ ,4} is the diagonal relation on A. 
(The reader should verify this.) 


What is the reflexive closure of the relation R = {(a, b) \ a < b} on the set of integers? 
Solution: The reflexive closure of R is 

R U A = {(a, b) | a < b) U {(a, a) \ a £ Z} = {(a, b) j a £ b}. ■ 


The relation {(1,1), (1,2), (2,2), (2,3), (3,1), (3,2)} on {1,2,3} is not symmetric. 
How can we produce a symmetric relation that is as small as possible and contains R’ 
To do this, we need only add (2,1} and (1,3), since these are the only pairs of the form 
(b, a) with (a, b) £ R that are not in R, This new relation is symmetric and contains 
R Furthermore, any symmetric relation that contains R must contain this new relation, 
since a symmetric relation that contains R must contain (2, 1) and (1, 3). Consequently, 
this new relation is called the symmetric closure of R. 

As this example illustrates, the symme tric closure of a relation R can be constructed 
by adding all ordered pairs of the form (b, a), where (a, b) is in the relation, that are not 
already present in R. Adding these pans produces a relation that is symmetric, that con¬ 
tains R, and that is contained in any symmetric relation that contains R, The symmetric 
closure of a relation can be constructed by taking the union of a relation with its inverse; 
that is, R U R 1 is the symmetric closure of R. where if -1 = {lb, A) | ia, b) £ R). The 
reader should verify this statement. 


What is the symmetric closure of the relation R - {(a, b) j a > b) on the set of positive 
integers? 

Solution: The symmetric closure of R is the relation 

R LJ R 1 = {ui, b) ' a > b) U {(b, a) | it > b\ - {(a, b) \ a A b}. ■ 


Suppose that a relation R is not transitive, How can we produce a transitive relation 
tiiat contains R such that this new relation is contained within any transitive relation that 
contains R1 Can the transitive closure of a relation R be produced by adding ail the pairs 
of the form {a, c) where (a, b) and (b, c) are already in the relation? Consider the relation 
R - {(], 3), (1,4), (2, 1), (3,2)} on the set {1,2, 3,4}. This relation is not transitive since 
it does not contain all pairs of the form (a, c) where (a, b) and (b, c ) are in R. The pairs 
of this form not in R are (1. 2), (2,3), (2,4), and (3, 1). Adding these pairs does not 
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EXAMPLE 3 


produce a transitive relation, since the resulting relation contains (3.1) and (1, 4) but 
does not contain (3,4). This shows that constructing the transitive closure of a relation 
is more complicated than constructing either the reflexive or symmetric closure. The 
rest of this section develops algorithms for constructing transitive closures. As will be 
shown, the transitive closure of a relation can be found by adding new ordered pairs 
that must be present and then repeating this process until no new ordered pairs are 
needed. 

PATHS IN DIRECTED GRAPHS 


We will see that representing relations by directed graphs helps in the construction 
of transitive closures. We now' introduce some terminology that we will use for this 
purpose. 

A path in a directed graph is obtained by traversing along edges (in the same di¬ 
rection as indicated by the arrow on the edge). 

DEFINITION 1. A peth ton a to b in the directed graph G is a sequence of 
one or more edges (xp, xj), {xi. xa), {x?, x 3 ),..., (x B _ j, x„) In G, where xp =* a and 
x„ = b, that is, a sequence of edges where the ter mi nal vertex of an edge is the 
same as the initial vertex in die next edge in the path. This path is denoted by 
xq. X[, x 2 , ■ ■ Xfl-p x„ and has length n. A path that begins aid ends at the same 
vertex is called a circuit or cycle. 


A path in a directed graph can pass through a vertex more than once. Moreover, an 
edge in a directed graph can occur more than once in a path. The reader should note 
that some authors allow paths of length zero, that is, paths consisting of no edges. In 
this book all paths must have a length of at least one. 


Which of the following are paths in the directed graph shown in Figure 1: a. b, e,d\ 
u. e. c. d. b; b, a, c, b. a, a, b; d, c ; c, b, a', e, b, a, b. a, h, el What are the lengths of those 
that are paths'? Which of the paths in this list are circuits? 

Solution: Since each of (a, b), (b, e), and (e, d) is an edge, a. b, e, d is a path of length 
three. Since {c, d) is not an edge, a, e, c. d. b is not a path. Also, b, a. c. b, a r a, b is a path 
of length six since (b. a), (fl, c), (r, b). (b, a), {a. a), and (a, b) are all edges. We see that 
d, c is a path of length one, since (d. cl is an edge. Also c, b, a is a path of length two. 



PDF created with pdfFactory trial version www.pdffactorv.com 







6,4 Cbsu res of Relations 399 


THEOREM 1 


EXAMPLE 4 


since (f, b) ami (b, a) are edges. All of (e, b) t (b, a), (a, b), ( b, a), ( a, b ), and ( h , e) are 
edges, so that e, t>, a, b, a, h, e is a path of length six. 

The two paths b, a, c, b, a, a, b and e, b, a, b, a, b, e tire circuits since they begin and 
end at the same vertex, The paths a, b, e, d: c, b, a: and d, c are not circuits. ■ 


The term path also applies to relations. Carrying over the definition from directed 
graphs to relations, there is a path from a to b in R if there is a sequence of dements a, 
xj, . ti . -hr-1- ^ with (a, JCi) G R, U|, a/t G and i>„ j, b ) G R. The follow¬ 

ing theorem can be obtained from the definition of a path in a relation. 


Let Abe a relation on a set A. There ia a path oflofgthV&ffi ^fetfiadealy if 

(it,b)€R n . '. 

Proof: We will use mathematical induction. By definition, there is a path from a to b 
of length one if and only if (a, h) £ R, so the theorem is true when n = I- 

Assume that the theorem is true for the positive integer n. This is the inductive 
hypothesis. There is a path of length a + 1 from a to b if and only if there is an element 
c £ ,4 such that there is a path of length one from a to c, so (a, c) £ R, and a path of 
length n from c to b, that is, (c, b) £ R n . Consequently, by the induction hypothesis, 
there is a path of length n + 1 from a to b if and only if there is an element c with 
(a, c) £ R and (c, b) £ R n . But there is such an element if and only if (a, b) £ R n * ! . 
Therefore, there is a path of length n + 1 from a to b if and only if (a, b) G R a * 1 , This 
completes the proof. □ 


TRANSITIVE CLOSURES 


We now 1 show 1 that finding the transitive closure of a relation is equivalent to determining 
which pairs of vertices in the associated directed graph are connected by a path With 
this in mind, we define a new relation. 


DEFINITION Z. Let # be a relation on a set A, The ctmmctivity nskttkm R" con¬ 
sists of the pairs (a, b) such that there is a path between aaa&bm R 


Since R n consists of the pairs (a. b) such that there ts a path of length n from a to 
h. it follows that R* is the union of ail the sets R", In other words, 

R t = jj R n . 

n -1 

The connectivity relation is useful in many models. 


Let R be the relation on the set of all people in the world that contains (a, h) if a has met 
b. What is R n where n is a positive integer greater than 2? What is R*7 

Solution: The relation R: contains [a, b) if there is a person c such that (a, c) G R and 
(r, b) G R, that is. if there is a person c such that a has met c and c has met b. Similarly, 
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R r ‘ consists of those pairs (a, b) such that there are people X[, t2> ■ ■ ■ > x n- 1 SLIt -'h that a 
has met .r, , jq has met Xj .and x„_i has met b. 

The relation R’ contains (a, b) if there is a sequence of people, starting with a and 
ending with b, such that each person in the sequence has met the nest person in the 
sequence. (There are many interesting conjectures about R\ Do you think that this 
connectivity relation includes the pair with you as the first dement and the president of 
Mongolia as the second element?) ■ 


Let R be the relation on the set of all subway stops in New' York City that contains (a, b) 
if It is possible to travel from stop a to stop b without changing trains. What is R n when 
n is a positive integer? What is R"1 

Solution: The relation R n contains (a, b) if it is possible to travel from stop a to stop b 
by making at most n — 1 changes of trains. The relation R' consists of the ordered pairs 
(a, b) where it is possible to travel from stop a to stop b making as many changes of 
trains as necessary. (The reader should verify these statements.) ■ 


Let R be the relation on the set of ail states in the United States that contains (a, h) if 
state a and state b have a common border. What is R n where n is a positive integer? 
What is r ? 

Solution: The relation R n consists of the. pairs (a, b ) where it is possible to go from state 
a to state b by crossing exactly n state borders. R* consists of the ordered pairs («, b) 
where it is possible to go from state a to state h crossing as many borders as necessary. 
(The reader should verify these statements.) The only ordered pairs not in R * are those 
containing states that are not connected to the continental United States (i.e., those pairs 
containing Alaska or Hawaii). B 


The following theorem shows that the transitive closure of a relation and the asso¬ 
ciated connectivity relation are the same. 


The transitive closure of a relation R equals the connectivity relation R*. 

Proof: Note that R* contains R. To show that R* is the transitive closure of R we must 
also show that /?* is transitive and that R* C S whenever S is a transitive relation that 
contains R. 

First, we show that R* is transitive. If (a, b) 6 R* and <b. c) t R\ then there are 
paths from a to b and from b to c in R. We obtain a path from a to c by starting with 
the path from a to b and following it with the path from b to c. Hence, {a, c)E R\ It 
follows that /?* is transitive. 

Now suppose that S is a transitive relation containing R Since S is transitive, 5" 
also is transitive (the reader should verify this) and S' 1 C S (by Theorem 1 of Section 
6.1). Furthermore, since 

(>■ 

* = i 
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and S k C S , it follows that S* C S. Now note that if R C S, then R r C ,S'\ because any 
path in R is also a path in S. Consequently. R" C S* C S. Thus, any transitive relation 
that contains R must also contain R'. Therefore, R' is the transitive closure of R, □ 

Now' that we know that the transitive closure equals the connectivity relation, we 
turn our attention to the problem of computing this relation. We do not need to exa min e 
arbitrarily long paths to determine whether there is a path between two vertices in a 
finite directed graph. As the following lemma shows, it is sufficient to examine paths 
containing no more than n edges, where n is the number of elements in the set. 


Let A be a set with n elements, and let if be a relation on A. If there is a path in R 
from a to b, then there is such a path with length not exceeding n. Moreover, when 
a # b, if there is a path in R from a to b, then there is such a path with length not 
exceeding n - 1. 

Proof: Suppose there is a path from a to b in R. Let m he the length of the shortest such 

path. Suppose that in, i|, x%, .i, where xq = a and x m — b, is such a path. 

Suppose that a = b and that m > n, so that m n+ i. By the pigeonhole principle, 

since there are n vertices in A, among the m vertices ,v 0 , jtj. x m ~\, at least two are 

equal (see Figure 2), 

Suppose that jq = xj with 0 ^ / < j ^ m— 1. Then the path contains a circuit 
from Xi to itself. This circuit can be deleted from the path from a to b, leaving a path, 
namely, xg, x h x„ r,+ 1( ..., x m - 1 , i m , from a to b of shorter length. Hence, the path 
of shortest length must have length less than or equal to 

The case where a t 4 b is left as an exercise for the reader, □ 

From Lemma L we see that the transitive closure of R is the union of R, R 2 . R 2 . 

and R n . This follows since there is a path in R" between two vertices ifand only if there 
is a path between these vertices in /?', for some positive integerwith i n. Since 

R’ -- R U R 2 U R 3 U * ■ ■ U R", 

and the zero-one matrix representing a union of relations is the join of the zero-one 
matrices of these relations, the zero-one matrix for the transitive closure is the join of 
the zero-one matrices of the first n powers of the zero-one matrix of R 


Let M# be the zero-one matrix of the relation R on a set with n elements. Then the 
zero-one matrix of the transitive closure R“ is 

M*. = M fi V Mj 1 V M^ 1 v * ■ * V Mjjl 
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Find the zero-one matrix of the transitive closure of the relation R where 


M* 


1 0 1 
0 1 0 
1 1 0 


Solution: From Theorem 3, it follows that the zero-one matrix of R * is 

M*. =M*V<\/Mgl 

Since 




1 1 1 
0 1 0 
I 1 I 


it follows that 


and 


Mg 1 - 


1 1 
0 1 
1 ! 


f 

0 

I 


1 0 f 


1 I 1] 

1 1 f 


'i i r 

0 1 0 

V 

0 1 0; \/ 

0 1 0 

= 

0 l 0 

1 1 0 


_1 1 ij 

1 1 1 


1 1 1 


Theorem 3 can be used as a basis for an algorithm for computing the matrix of the 
Wl$ relation R*. To find this matrix, the successive Boolean powers of Mg, up to the mh 

• 1 power, are computed. As each power is calculated, its join with the join of all smaller 

powers is formed. When this is done with the nth power, the matrix for R* has been 
found. This procedure is displayed as Algorithm 1. 


ALGORITHM l A Procedure for Computing the Transitive Closure, 
procedure transitive closure (Ms : zero-one n X n matrix) 


A : = 

Mg 


B : = 

A 


for i ; 

= 2 

to n 

begin 



A; 

= A 

0 Mj 

B : 

= B 

VA 


end {B is the zero-one matrix for 1?*} 


We can easily find the number of bit operations used by Algorithm 1 to determine the 
transitive closure of a relation. Computing the Boolean powers Mg, ..., re¬ 
quires that n - 1 Boolean products of nXn zero-one matrices be found. Each of these 
Boolean products can be found using n 2 (ln - t) bit operations. Hence, these products 
can be computed using n 2 (2n - Din - 1) bit operations. 

To find M;,. from the n Boolean powers of Mg, n - 1 joins of zero-one matrices 
need to be found. Computing each of these joins uses >r bit operations. Hence, (n - 1 )rr 
bit operations are used in this part of the computation. Therefore, when Algorithm I is 
used, the matrix of the transitive closure of a relation on a set with n elements can 
be found using rr(2n - l)(n - 1} + (n - l pr = 2nVi - 1) = 0(« 4 ) bit operations. 
The remainder of this section describes a more efficient algorithm for finding transitive 
closures. 
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WARSHAIJ/S ALGORITHM 

WarshalFs algorithm, named after Stephen Warshall, who described this algorithm in 
1960. is an efficient method forcomputing the transitive closure of a relation. Algorithm 
1 can fi nd the transitive closure of a relation on a set with n elements using In-(a - 1) 
bit operations. However, the transitive closure can be found by WarshalFs algorithm 
using only 2 n } bit operations. 

Remark: Warshall’s algorithm is sometimes called the Roy-Warshall algorithm, since 
B. Roy described this algorithm in 1959. 

Suppose that R is a relation on a set with n elements. Let v,, _ v„ be an ar¬ 

bitrary listing of these n dements. The concept of the interior vertices of a path is 
used in Warshall’s algorithm. If a, x,, xi ,..., x m _ i , bis a path, its interior vertices are 
x\, X 2 , ■ ■ ■. x m ], that is, all the vertices of the path that occur somewhere other than 
as the first and last vertices in the path. For instance, the interior vertices of a path 
a, c, d, /, g. h, b, j in a directed graph are c, d, f, g, h. and h The interior vertices of 
a, v. d, u, f, b are c, d, a t and f. (Note that the first vertex in the path is not an interior 
vertex unless it is visited again by the path, except as the last vertex. Similarly, the last 
vertex in the path is not an interior vertex unless it was visited previously by the path, 
except as the first vertex.) 

WarshalFs algorithm is based on the construction of a sequence of zero -one ma¬ 
trices. These matrices are W f) , W,,..., W„, where VV 0 = M g is the zero-one matrix 
of this relation, and Wj. = [w'jy ], where = l if there is a path from v, to Vj such 
that all the interior vertices of this path are in the set {v L , v 2 ,.... v k ) (the first k vertices 
in the list) and is 0 otherwise. (The first and last vertices in the path may be outside 
the set of the first k vertices in the list.) Note that W„ = M r , since the (;, j )th entry 
of Mjp is 1 if and only if there is a path from v, to v,, with all interior vertices in the 
set {i-i, m,..., v„} (but these are the only vertices in the directed graph). The following 
example illustrates what the matrix W i: represents. 


Stephen Warshall (born 19351. Stephen Warshall, bom in New York City, wont to public school in 
Brooklyn, He attended Harvard University, receiving his degree in mathematics in 1956. He never received 
an advanced degree, since at that time no programs were available in his areas of interest. However, he took 
graduate courses at several different universities and contributed to the development of compuier science 
and software engineering, 

After graduating from Harvard, Warshall worked at ORO (Operation Research Office), which was 
set up by Johns Hopkins to do research and development for the U.S, Army. In 1958 he left ORO to take a 
position at a company called Technical Operations, where he helped build a research and development labo¬ 
ratory for military software projects. In 1% I he left Technical Operations to found Massachusetts Computer 
Associates. Later, this company became part of Applied Data Research (ADR), After the merger, Warshall 
sat on the hoard of directors of ADR and managed a variety of projects and organizations. He retired from 
ADR in 1982 

During his career Warshall carried out research and development in operating systems, compiler de¬ 
sign, language design, and operations research, in the J 97! — 1972 academic year he presented lectures on 
software engineering at French universities. There is an interesting anecdote about his proof that the tran¬ 
sitive closure algorithm, now' known as WarshalFs algorithm, is correct. He and a colleague at Technical 
Operations bet a bottle of rum on who first could determine whether this algorithm always works. Warshall 
came up with his proof overnight, winning the bet and the rum, which he shared with the loser of the bet. 
Because Warshall did not like sitting at a desk, he did much of his creative work in unconventional places 
such as bn a sailboat in the Indian Ocean or in a Greek lemon orchard, 
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f IGF RE 3 The Directed Graph of the Relation R. 


K X Let R be the relation with directed graph shown in Figure 3. Let a, b. d be a listing 

of the dements of the set. Find the matrices W<], Wj, WL, YVt, and VV 4 . The matrix W 4 
is the transitive closure off?. 


Solution: Let V[ - a, vj = b, vi = r, and ty - d. Wy is the matrix of the relation. 
Hence, 


W tJ 


0 0 0 11 
1 (j 1 0 ! 
1 00 1 ! 
0 0 1 oj 


IT] has 1 as its u, / )th entry if there is a path from u, to v, that has only ij - a as 
an interior vertex. Note that all paths of length one can still be used since they have 
no interior vertices. Also, there is now an allowable path from b to d, namely, b.a, d. 
Hence, 


0 OOll 
1 0 1 i! 
1 0 0 1 : 

L 0 0 J Oj 


W-> has 1 as its (i, j )th entry if there is a path front v, (u v, that has only v, - a and/or 
v ; - b as «s interior vertices, if any, Since there are no edges that have b as a terminal 
vertex, no new paths are obtained when we permit b to be an interior vertex. Hence, 

W W 

W has 1 as its (i, /Jth entry if there is a path from vy to ij- that has only f, - a, 
t’2 - h, and/or vy = c as its intenor vertices, if any. We now have paths from d to a, 
namely, d,c, a, and from d to d. namely, d, c, d. Hence, 


[0 0 0 ( 1 
; 10 11 

I 1 0 0 I I 

[1 0 1 ij 


Finally, W 4 has 1 as its. (i, jjth entry if there is a path from v,- to vy that has ty = a. 
v-i = b, ij - r, and/or V 4 = d as interior vertices, if any. Since these are all the vertices 
of the graph, ihis entry is 1 if and only if there is a path from ty to v f . Hence. 
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LEMMA 2 


W 4 


10 11 
10 11 
10 11 
10 11 


This last matrix, W4, is the matnx of the transitive closure. 


Warshali's algorithm computes by efficiently computing Wo = M*Wj, 
Wj.,..,W„ = Mp. The following observation shows that we can compute 
W t directly from W*-i: There is a path from vy to v, with no vertices other than 
i’i, vi,. ,., as interior vertices if and only if either there is a path from vy to vj 
with its interior vertices among the first k — 1 vertices in the list, or there are 
paths from Vj to v* and from v* to r, that have interior vertices only among the first 
k - 1 vertices in the list. That is, either a path from f, to ;y already existed before 
i-i was permitted as an interior vertex, or allowing v* as an interior vertex produces 
a path that goes from vy to v* and then from \\ to vy. These two cases are shown in 
Figure 4. 

The first type of path exists if and only if w^ 1 - = 1, and the second type of path 
exists if and only if both wj* ^ and vrjf : are 1. Hence, wjj 1 is 1 if and only if either 
wj;- 1 ' is 1 or both wjj -11 and wjj" 11 are 1, This gives us the following lemma. 


Let W* = [wj; 1 ] be the zero-one matrix that has a 1 in its (f, y)th position if and 
only if there is a path from v/ to vy with interior vertices from the set (vp vj,..., v*}. 
Then 

< ] = 

whenever i,j, and k are positive integers, not exceeding n. 


All interim vertices 

in {i|,Vj-v t ,} 





FIGURE 4 Adding v* to the Set of Allowable Interior Vertices. 
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Lemma 2 gives us the means efficiently to compute the matrices W t , k - 

1,2 . n. We display the pseudocode for WarshalPs algorithm, using Lemma 2, 

as Algorithm 2, 


ALGORITHM 1 Wars hall Algorithm. 

procedure Warshaii (M/f : n X n zero-one matrix) 

W := 

for k : - 1 to n 
begin 

for ; : = 1 to n 
begin 

for / ; = 1 to n 

Wij := WijSf(yVi k Aw k j) 

end 

end {W - [w u ] is M r } 


The computational complexity of WarshalPs algorithm can easily be computed in terms 
of bit operations. To find the entry w\f from the entries wjj 1 wj* - n . and 11 us¬ 
ing Lemma 2 requires two bit operations. To find all n 2 entries of W* from those of 
W*. [ requires in 2 bit operations. Since WarshalPs algorithm begins with W n - M* 
and computes the sequence of n zero-one matrices W t , Wg,..., W„ = M A -, the total 
number of bit operations used is n ■ 2 n~ - 2n 3 . 


Exercises 


L Let R be the relation on the set {0, !, 2. 3} containing 
the ordered pairs jO, I), j 1, 1), (1,2), (2,0), (2,2), and 
(3.0) Find the 

a) reflexive closure of R h) symmetric closure of /?, 

2. Let R be the relation {(a, b) \ a ^ h\ on the set of inte¬ 
gers What is the reflexive closure of/?? 

3. Let R he the relation {{a, b ) | a divides b} on the set of 
integers, What is the symmetric closure of A’? 

4. How can the directed graph representing the reflexive 
closure of a relation on a finite set be constructed from 
the directed graph of the relation? 

In Exercises 5-7 draw the directed graph of the reflexive 
closure of the relations with the directed graph shown. 




8, How can the directed graph representing the symmetric 
closure of a relation on a finite set he constructed from 
the directed graph for this relation? 

9. Find the directed graphs of the symmetric closures of 
the relations with directed graphs shown in Exercises 
5-7. 

10. Find the smallest relation containing the relation in 
Example 2 that is both reflexive and symmetric 

11. Find the directed graph of the smallest relation that is 
both reflexive and symmetric for each of the relations 
with directed graphs shown in Exercises 5-7. 

12. Suppose that the relation R on the finite set A is rep¬ 
resented by the matrix M s . Show that the matrix that 
represents the reflexive closure of R is M R v / \ r 
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13. Suppose that file relation A on the finite set A is rep¬ 
resented by the matrix M s , Show that the matrix that 
represents the symmetric closure of A is M* v MJ,. 

14. Show that the closure of a relation R with respect to 
a property P, if it exists, is the intersection of all the 
relations with property P that contain R 

15. When is it possible to define the “irreflexive closure'' 
of a relariou A, thar is, a relation that contains R, is ir- 
retJexivc, and is contained in every irreflexive relation 
that contains, R' 

1 fi. Determine whether the fallowing sequences of vertices 
are paths in the following directed graph. 



ai a.b.t.e 

b) b. v,h, c 

c) (i. it, b, p, (I, e 

d) h. r, p, d, a, a. b 
c) h.i\c.b.e,d, i. d 
f) a, a. b, h. r. c, b. e, d 

17. Find all circuits of length three in the directed graph in 
Exercise I ft. 

18. Determine whether there is a path in the directed graph 
in Exercise 16 beginning at the lirsi vertex given and 
ending ut the second vertex given. 

a) it, h 
ll) f \ d 

c) h. b 

d) U. r 
t! b.d 
n v.d 
s) d. d 

h) c, a 

i) «. C 

19. 1 .el R be the relation on the set {l, 2, 3,4, 5} containing 

the ordered pairs (1, 3.1, (2, 4), (3, 1), (3. Vj, (4,3), (5, I), 
(5, 2), and (5, 4). Find 

a) R- b) A\ 

ci ir d) R-. 

e) R - f) Kb 

20. Let R be the relation that contains the pair (a, b)ifa and 
b are cities such that there is a direct non-stop airline 
flight from a to h. When is [a. b) in 

a) R : " 

b) R s ? 
ci A”’ 

21. Let R be the relation on the set of all students contain¬ 
ing the ordered pair (u, b) if« and b are in at least one 


common class and a ^ b. When is (a, b) in 

a) A 1 ? 

b) A’ ! 1 
ci A"? 

22. Suppose that the relation A is reflexive. Show that R' 
is reflexive. 

23. Suppose that tlic relation R is symmetric. Show that R‘ 
is symmetric, 

24. Suppose that the relation A is irreflexive. Is the relation 
A 1 necessarily irreflexive"? 

25. Use Algorithm 1 to find the transitive closures of the 
following relations oil {], 2, 3,4}. 

a) {(E2). (2, 1), (2,3). (3,4), (4, ))} 

b) {(2, 1), (2,3), (3,1), (3,4), (4, J).(4,3)} 

C) {(1,2), (I,3), (1,4), (2, 3), (2,4). (3,4)} 

d) {{1,1), (1,4), (2,1). (2,3), (3,1), (3,2), (3,4). (4,2)} 

26. Use Algorithm 1 to find the transitive closures of the 
following relations on {a, b, c, d, t\. 

a) {(a, c). (b, d), (r, a), (d, b), (e, d)} 

b) l(f>, c), (b, e), (c, e), (d, a), {e, b ), it. c)} 

c) {(a, *). (a. <:), (a. t), ( b , a), ( b . c), (c. a), (c, b). 

(d, a), (e, d)J 

d) {(«. e), {b, a), (b, d\ (c. d), f d, a), ( d, c). (e, a), 

(e, b), (c, c\ (e, e)} 

27. Use Warshall’s algorithm to find the transitive closures 
of die relations in Exercise 25. 

28. Lfse Warshalt’s algorithm to find the transitive closures 
of the relations in Exercise 26. 

29. Find the smallest relation containing the relation 
{(!. 2), (1,4), (3,3), (4, 1)} that is 

a) reflexive and transitive. 

b) symmetric and transitive. 

c) reflexive, symmetric, and transitive. 

30. Finish the proof of the ease when a ^ bin Lemma 1. 

31. Algorithms have been devised that use (Tfn 2 ® I bit op¬ 
erations to compute the Boolean product of two n x n 
zero-one matrices. Assuming that these algorithms can 
be used, gi ve big-D estimates for the number of bit op¬ 
erations using Algorithm I and using Warshall’s algo¬ 
rithm to find the transitive closure of a relation on a set 
with tt elements. 

*32, Devise an algorithm using the concept of interior ver¬ 
tices in a path to find the length of the shortest path be¬ 
tween two vertices in a directed graph, if such a path 
exists. 

33. Adapt Algorithm 1 to find the reflexive closure of the 
transitive closure of a relation on a set with n elements. 

34. Adapt Warshail’s algorithm to find the reflexive closure 
of the transitive closure of a relation on a set with n 
elements, 

35. Show that the closure with respect to the property P of 
the relation A = {(0,0), (0, [),(], 1), (2, 2)) on the set 
{0,1,2) does not exist if P is the property 

a) “is not reflexive.” 

b) “has an odd number of elements." 
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6.5 __ __ 

Equi v ale ncc Re 1 at ions 

INTRODUCTION 

Students at a college register for classes the day before the start of a semester. Those 
with last names beginning with a letter from A to G, front H to N, and from 0 to 
Z may register at any time in the periods 8 a m. in 11 a.m., 11 a.m. to 2 PM., and 
2 pm to 5 pm., respectively. Let R be the relation containing (a:, y) if and only if .v 
and y are students with last names beginning with letters in the same block. Conse¬ 
quently, x and y can register at the same time if and only if (a, y) belongs to R. It is 
easy to see that R is reflexive, symmetric, and transitive. Furthermore, R divides 
the set of students into three classes, depending on the first letters of their last 
names. To know when a student can register we are concerned only with which 
of the three classes the student is in, and we do not care about the identity of the 
student. 

The integers a and b are related by the “congruence modulo 4” relation when 4 
divides a-b. We will show later that this relation is reflexive, symmetric, and transitive. 
It is not hard to see that a is related to b if and only if a and h have the same remainder 
when divided by 4. It follows that this relation splits the set of integers into four different 
classes. When we care only what remainder an integer leaves when it is divided by 4, 
we need only to know which class it is in, not its particular value. 

These two relations, R and congruence modulo 4, are examples of equivalence 
relations, namely, relations that are reflexive, symmetric, and transitive. In this section 
we will show that such relations split sets into disjoint classes of equivalent elements. 
Equivalence relations arise whenever we care only whether an element of a set is in a 
certain class of elements, instead of caring about its particular identity' 


EQUIVALENCE RELATIONS 


In this section we w ill study relations with a particular combination of properties that 
allows them to be used to relate objects that are similar in some way. 

DEFINITION 1, A relation on a set A is called an equivalence relation if it is 
reflexive, symmetric, and transitive. 


Two elements that are related by an equivalence relation are called equivalent 
(This definition makes sense since an equivalence relation is symmetric.) Since an 
equivalence relation is reflexive, in an equivalence relation every demerit is equiva¬ 
lent to itself. Furthermore, since ati equivalence relation is transitive, if a and b are 
equivalent and h and c are equivalent, it follows that a and c are equivalent. 

The following examples illustrate the notion of an equivalence relation. 


EXAMPLE 1 Suppose that R is the relation on the set of strings of English letters such that a R b 

if and only if 1(a) = l{b), w here l(x') is the length of the string x. Is R an equivalence 
relation? 
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EXAMPLE 2 


EXAMPLE .3 


EXAMPLE 4 


Solution: Since 1(a) = 1(a ), il follows that a R a whenever a is a string, so that R is 
reflexive. Next, suppose that a R b, so that 1(a) = 1(b). Then b R a, since 1(b) = 1(a). 
Hence, R is symmetric. Finally, suppose that a R b and b R c. Then ((a) — 1(b) and 
1(b) = 1(c ), Hence, 1(a) - 1(c), so that a R c. Consequently, R is transitive. Since R is 
reflexive, symmetric, and transitive, it is an equivalence relation, ■ 


Let R be the relationonthesetofintegerssuchthala/fi)ifandonly if a = bora - -b. 
In Section 6,1 we showed that R is reflexive, symmetric, and transitive. It follows that 
R is an equivalence relation. ■ 


Let R he the relation on the set of real tmmhers such that a R b if and only if a - b is 
an integer, is R an equivalence relation? 

Solution: Since a-a = 0 is an integer for all real numbers a,aRa tor all real numbers 
a. Hence, R is reflexive. Now suppose that a R b. Then a- b is an integer, so that b - a 
is also an integer. Hence, b R a. It follows that If is symmetric. If a R b and b R c. then 
a - b and h ~ c are integers. Therefore, a - c = (a - b) + (b - c) is also an integer. 
Hence, a R c. Thus, R is transitive. Consequently, R is an equivalence relation, ■ 


One of the most widely used equivalence relations is congruence modulo m, where 
m is a positive integer greater than 1. 


Congruence Modulo tn Let m be a positive integer greater than 1. Show that the 
relation 

R - {(u b) | a ™ b (mod m)} 
is an equivalence relation on the set of integers. 

Solution: Recall from Section 2.3 that a = b (mod m) if and only if m divides a - b. 
Note that a - a ~ 0 is divisible by in. since 0 = 0* m. Hence, a ^ a (mod m), so 
that congmence modulo m is reflexive. Now suppose that a = 6 (mod in). Then a b is 
divisible by m. so that a-h - km, where k is at integer. It follows that ha - ( -k)m, 
so that (mod m). Hence, congruence modulo m is symmetric. Next, suppose that 
o = 6 (mod m) and b = c (mod m). Then m divides both a ~ b and b - t. Therefore, 
there are integers k and / with a-b ~ km and b - c - Im. Adding these two equations 
shows that a c ~ (a - h) + (b - c) = km + Im = (k + l)m. Thus, a s= c (mod m). 
Therefore, congruence modulo m is transitive. It follows that congruence modulo m is 
an equivalence relation, ■ 

EQUIVALENCE CLASSES 


L.et A be the set of all students in your school who graduated from high school. Consider 
the relation R on A that consists of all pairs f.v, y) where x and y graduated from the same 
high school. Given a student x. we can form the set of all students equivalent to x with 
respect to R. This set consists of all students who graduated from the same high school 
as x did. This subset of A is called an equivalence class of the relation. 
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EXAMPLE 5 


EXAMPLE (> 


VfnNmN:& La R fac its eqiMvafen.ee relation on a set A. The set of all ele- 
SB ekstnwt a of A is caJtedthe equivalence class of a. The 
eqjufti&& &hk of is tvHh reelect to R is-denoted by [d}*.‘Wben only one relation is 
andercorlsideration, we will delete the subscript R and write fa] for this equivalence 
class. 

In other words, if R is an equivalence relation on a sel A, the equivalence class of the 
dement a is 

[tils - (c j ( t j, .r) e R}. 

It b £T [a]^, b is called u representative of this equivalence class. 


What, is the equivalence class of an integer for the equivalence relation of Example 2° 

Solution: Since ail integer is equivalent to itself and its negative in this equivalence 
relation, it follows that [a] = {-a,a}. This set contains two distinct integers unless 
a - 0. For instance, [7J = {-7, 7}. [ 5J = {-5. 5f, and JO] = {0}. ■ 


What arc the equivalence classes of 0 and I for congruence modulo 4? 

Solution: The equivalence class of 0 contains all integers a such that a = 0 (mod 4j, 
The integers in this class are those divisible by 4. Hence, the equivalence class of 0 tor 
this relation is 

ID] - {....-8, -4.0,4, 8,.. 

The equivalence class of 1 contains all the integers a such that a = 1 (mod 4). The 
integers in this class are those that have a remainder of I when divided by 4. Hence, 
the equivalence class of l for this relation is 

[ 1 ] = {...,-7,-3, 1 , 5 , 9 ,..,}. | 


in Example 6 the equivalence classes of 0 and 1 with respect to congruence modulo 
4 were found. Example 4 can easily be generalized, replacing 4 with any positive integer 
m. The equivalence classes of the relation congruence modulo m are called the congru¬ 
ence classes modulo m. The congruence class of an integer a modulo m is denoted by 
\a\ m so that [a] m = {..., a - 2m, a - in, a, a 4- m, a + 2m, .. For instance, from Exam¬ 
ple 6 it follows that [0] 4 = -8, -4. 0,4, 8,...} and [ 1] 4 = {-7, - 3 , 1 , 5 , 9....}. 

EQUIVALENCE CLASSES AND PARTITIONS 


Let A be the sol uf students at your school who are majoring in exactly one subject, and 
let R be the relation on A consisting of pairs (a, y) where x and y are students with the 
same major. Then R is an equivalence relation, as the reader should verify. We can see 
that R splits all students in A into a collection of disjoint subsets, where each subset 
contains students with a specified major. For instance, one subset contains all students 
majoring (just) in computer science, and a second subset contains all students major¬ 
ing in history. Furthermore, these subsets are equivalence classes of R. This example 
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THEOREM I 


illustrates how the equivalence classes of an equivalence relation partition a set into 
disjoint, nonempty subsets. We will make these notions more precise in the following 
discussion. 

Let K be a relation on the sel .4, Hie following theorem shows that the equivalence 
classes of two elements of A are either identical or disjoint. 


Let R be an equivalence relation on a set A. The following state ments are equivalent: 

(i) aRb 

(ii) M = [b] 
fiii) [a] n [b] r 0 

Proof: We first show that (i) implies (ii), Assume that a R b. We will prove that fa] = 

I hi by showing |a) c f b] and |i>] C fa], Suppose c E [a]. Then a R c. Since a R hand 
R is symmetric, we know that b R a. Furthermore, since R is transitive and b R a and 
a R r, it follows that b R c. Hence, c £ [h]. This shows that [a] C [b]. The proof that 
\b\ c fa] is similar; it is left as an exercise for the reader. 

Second, we will show that (ii) implies {iii). Assume that faj - \b]. It follows that 
[«i H fri] s 6 0 since [c] is nonempty (since a E fa] because R is reflexive). 

Next, we will show that (iii) implies (i). Suppose that fa] n Jh] # 0. Then there 
is an element c with e E [af and c £ [h]. In other words, a R r and b R c. By the 
symmetric property, c Rb. Then by transitivity, since a R c and t R h, we have a R b. 

Since (i) implies DU, (ii) implies (iii), and (iii) implies (i), the three statements, fi), 
(ii), and (iii), are equivalent. □ 

Wc are mm- in a position to show how an equivalence relation parlititms a sel. Let 
R be an equivalence relation on a set 4. The union of the equivalence classes of R is 
all of 4, since an element a of 4 is in its ow n equivalence class, namely, [a]*. In other 
words, 

U 1*1* = A. 

uE A 

In addition, from Theorem 1, it follows that these equivalence classes are either equal 
or disjoint, so 

[a] w n 1 * 1 * = 0 

when [fljfi 7 s [&]«. 

These two observations show that the equivalence classes form a partition of 4, 
since they split 4 into disjoint subsets. More precisely, a partition of a set S is a col¬ 
lection of disjoint nonempty subsets of S that have S as their union. In other words, 
the collection of subsets 4,, i E I (where / is an index set) forms a partition of S if 
and only if 

4, ^ 0 for i E /. 

A, fl 4 j - 0, when i A j, 

and 

U-c-s. 

IE i 

(Here the notation U,;g/4/ represents the union of the sets 4, for all i E /.) Figure 1 
illustrates the concept of a partition of a set. 
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EXAMPLE 7 


THEOREM 2 


EXAMPLE 8 



Suppose that S' = {1, 2, 3,4,5, 6 }. The collection of sets A i = {1,2.3}, A 2 - {4, 5}, and 
At = { 6 } forms a partition of 5, since these sets are disjoint and their union is $- ■ 


We have seen that the equivalence classes of am equivalence relation on a set form a 
partition of the set. The subsets in this partition are the equivalence classes. Conversely, 
ever ) 1 partition of a set can be used to form an equivalence relation. Two elements are 
equivalent with respect to this relation if and only if they are in the same subset of the 
partition. 

To see this, assume that {A, \ i £ /} is a partition on S. Let R be the relation on S 
consisting of the parr (x, v) where x and v belong to the same subset A, in the partition. 
To show that R is an equivalence relation we must show that R is reflexive, symmetric, 
and transitive. 

We see that (a, a) £ R for every a £ S, since a is in the same subset as itself. 
Hence. R is reflexive. If (a, b) £ R, then b and a are in the same subset of the partition, 
so that (b, a) £ R as well. Hence, R is symmetric. If (a, b) £ R and (b. c) £ R, then a 
and b are in the same subset in the partition, X, and b and c are in the same subset of 
the partition, Y. Since the subsets of the partition are disjoint, and h belongs to X and Y, 
it follow s that X = Y. Consequently, a and c belong to the same subset of the partition, 
so that (a, t:) £ R. Thus, R is transitive 

It follows that R is an equivalence relation. The equivalence classes of R consist 
of subsets of S containing related elements, and by the definition of R, these are the 
subsets of the partition. Theorem 2 summarizes the connections we have established 
between equivalence relations and partitions. 


Let R be an equivalence relation on a set S. Then the equivalence classes of R form 
a partition of 5, Conversely, given a piartition {A,-1 i E 1} of the set X, there is an 
equivalence relation R that has the sets A;, i £ /, as its equivalence classes. 


The congruence classes modulo m provide a useful illustration of Theorem 2. There 
are m different congruence classes modulo m, corresponding to the m different remain¬ 
ders possible when an integer is divided by m. These m congruence classes arc denoted 
by [0] m , fljm.[m - ]]„,. They form a partition of the set of integers. 


What arc the sets in the partition of the integers arising from congruence modulo 4? 
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Solution: There are four congruence classes, corresponding to (0]4, [1]4, [ 2 J 4 , and PJ 4 . 
They are the sets 

LOU t . 8. -4,0, 4,8,,..} 

[Ik - {...,-7,-3,1,5,9,.,,} 

[2] t = {...,- 6 ,- 2 . 2 , 6 , 10 ,...} 

|3k - {...,-5, -1.3,7,11....}. 

Thesc congruence classes are disjoint, and every integer is in exactly one of" them. In 
other words, as Theorem 2 says, these congruence classes form a partition. ■ 


Exercises 


1. Which nf the following relations on {0,1,2,3} are 
equivalence relations'/ Determine the properties of an 
equivalence relation that the others lack. 

ai f(tt0i, (1.IU2, 2), (3.3)} 

h) j(0. (t). (0. 2 1 , (2, 0), (2. 2). (2, 3). (3, 2), (3, 3)} 

c) itftOi.tl, l).t l. 2),{2, l).a. 2),(3, 3)} 

(!) itaoui, IUl,3),a2j,C2,3j,aij 1 (3,2), (3,3)} 
ej /(C). 111,(0. 11,(0. 2), (1.0), (1, Iff I, 2), (2,0), 

(2. 2u3, 3)} 

2. Which >>f xlic following relations on the set of all people 
are equivalence relations'/ Determine the properties of 
an equivalence relation that the others lack. 

a) it li. b) | « and b are the same age} 

b) [tii, h 1 j a and b have the same parents} 
r) />) I 11 and b share a common parent} 
tit he, h) (j and b have met} 

e) jfo. h) | a and b .speak a common language} 

3. Which of the following relations on the set of all func¬ 
tions from Z to Z are equivalence relations? Determine 
the properties of an equivalence relation that die others 
lack. 

a) !(/.,i>) | /(I) - .?(!)} 

h) if/', y) [ /(0) - ,y(0) or/(1) » |f(l)} 

d [(/. <>) 1 /(.»•) - s!(.v) = I for all a e Z} 

d) {(t «) | /f.r! - t*t-v) - C for some C E Z for all 
.tf 7 ' 

e) !■;/, y 1 j /(0) ■ y( I) and /< 1) --- y(0)} 

4. Deltne three equivalence relations on the set of students 
in vDiu discrete mathematics class different from die 
relations discussed in the test. Determine the equiva¬ 
lence classes for these equivalence relations. 

5. Suppose that A is a nonempty set, and/ is a function 
that has A as its domain, Let R be the relation on A con¬ 
sisting of all ordered pairs (a, yl where /(,t) = f(y), 
at Show (hat R is an equivalence relation on .4. 

It) What are the equivalence classes of fl? 

6. Suppose that A is a nonempty set and R is an equiv¬ 
alence relation nn .4. Show that there is a function/ 
-.villi \ as us domain such that u, v) € R if and only if 

/is' ?t, VI. 


7. Show that the relation J?, consisting of all pairs (r, y) 
where x andy are hit strings nf length three or more that 
agree in their first three bits, is an equivalence relation 
on the set of all hit strings of length three or more. 

8. Show that the relation R, consisting of all pairs (x, v) 
where ,t and y are bit strings of length three or more 
that agree except perhaps in their first three bits, is an 
equivalence relation on the set of all bit strings, 

9. Show 1 that propositional equivalence is an equivalence 
relation on the set of all compound propositions. 

10. Let R be the relation on the set of ordered pairs of posi¬ 
tive integers such that ((.j, hi, (c, d)) G R if and only if 
ad = he. Show that R is an equivalence relation. 


In Exercises 11-13 determine whether the relation with the 
directed graphs show n is an equivalence relation. 



14. Determine whether the relations represented by the fol¬ 
lowing zero-one matrices are equivalence relations. 


r 1 1 n 

1 0 

1 o' 


111 

0 1 

0 1 


JZ' 

0 - 

1 0 

1 0 

c) 


0 1 

0 ij 



PDF created with pdfFactory trial version www.pdffactorv.com 








414 6 Relations 


15. Show that the relation R on the set of all bit strings such 
that s R i if and only it ,v and t contain the same number 
of Is is an equivalence relation. 

It). What are the equivalence classes of" the equivalence 
relations in Exercise ! ? 

17. What are the equivalence classes of the equivalence 
relations in Exercise 2" 1 

18. What are the equivalence classes of the equivalence 
relations in Exercise .V’ 

19. Whol is the equivalence class of the bit string 0) 1 for 
the equivalence relation in Exercise 15? 

2(1. What are the equivalence classes of the following bit 
strings for the equivalence relation in Exercise 7? 
ai 010 b) 1011 c) Hill d) 01010101 

21, Describe the equivalence classes of the hit strings in 
Exercise 20 far the equivalence relation from Exer¬ 
cise S, 

22. What i.s the congruence class [4]® when m is 

a' 2? hi 4? < ! 6? d) 8 ’ 

2d. Give a description of each of the congruence classes 
modulo 6. 

24. a) What is the equivalence class of (1, 2) with respect 

to the equivalence relation in Exercise 10? 
hi Give an interpretation of the equivalence classes for 
the equivalence relation R in Exercise 10. 

25. Which of the following collections of subsets are parti¬ 
tions of {I, 2, 3.4,5.6}? 

a) {1, 2f, {2. 3,4}. {4.5, 6f b{ 3, 6}, {4}, {5} 
c> {2.4.6}, {1.3.5} d) {1,4,5}, {2.6} 

2fi. Which of the following collections of subsets are parti¬ 
tions of the set of integers'.' 

a) the set of even integers and the set of odd integers 

b) Lbe set of positive integers and the set of negative 

integers 

c) the set of integers divisible by 3. the set of integers 
leaving a remainder of 1 when divided by 3, and 
the set of integers leaving a remainder of 2 when 
divided by 3 

d 1 the set of integers less than -100, the set of integers 
with absolute value not exceeding 100, and the set 
of integers greater than !(X) 
of the set of integers not divisible by 3, the set of even 
integers, and the set of integers ihar leave a remain¬ 
der id" 3 when divided by 6 

A partition is called a refinement of the partition P 2 if 
every set in P\ is a subset of one of the sets in P : . 

27. 5how that the partition formed from the congruence 
classes modulo 6 is a refinement of the partition formed 
from the congruence classes modulo 3. 

28. Suppose that A'i and AS are equivalence relations on a 
set A. let Pi and P-> be the partitions that correspond 
to A’| and A;, respectively. Show that A, C U 2 if and 
only if P is u refinement of /V 


29. Find the smallest equivalence relation on the set 
{a, b, c, d. e) containing the relation {fa, b), (a, c). 

(A < •}. 

30. Suppose that A, and AS are equivalence relations on the 
set S. Determine whether each of the following combi¬ 
nations of A, and R 2 must be an equivalence relation, 
a) Ai U R, b> A] n R 2 c) A, ® A; 

31. Consider the equivalence relation from Example 3, 
namely, A = {(jt, y) | x ~ y is an integer}. 

a) What is the equivalence class of I for this equiva¬ 
lence relation? 

b) What is the equivalence class of 1/2 for this equiv¬ 
alence relation? 

*32. Each bead on a bracelet with three beads is either red, 
white, or blue, as illustrated in the example shown. 
Define the relation R between bracelets as follows: 
(Bj, /El. where B j and B 2 are bracelets, belongs to A if 
and only if Hi can be obtained from B\ by rotating it or 
rotating d and then reflecting it, 


Bead I 
Red 



a) Show that A-is an equivalence relation. 

b) What arc the equivalence classes of A? 

*33. Let A be the relation on the set of all colorings of the 2 x 
2 chessboard where each of the foursquares is colored 
either red or blue so that (Ci, Ci), where Ci and C 2 
are 2x2 chessboards with each of their four squares 
colored blue or red, belongs to R if and only if C? can 
be obtained from C] either by rotating the chessboard 
or by rotating it and Lhen reflecting it. 

a) Show that A is an equivalence relation. 

b) Wha! are the equivalence classes of A? 

34. a) Let A be the relation on the set of functions from Z“ 

to Tp such that (/, g) belongs to A if and only iff 
is 0(g) [see Section 1,8). Show that A is an equiv¬ 
alence relation. 

b! Describe the equivalence class containing f(n) - 
rr for the equivalence relation of part (a). 

35. Determine the number of different equivalence rela¬ 
tions on a set with three elements by listing them. 

36. Determine the number of different equivalence rela¬ 
tions on a set with four elements by listing them, 

*37. Do we necessarily get an equivalence relation when we 
form the transitive closure of the symmetric closure of 
the reflexive closure of a relation? 

*38. Do we necessarily get an equivalence relation when we 
form the symmetric closure of the reflexive closure of 
the transitive closure of a relation? 
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39. Suppose we use Theorem 2 to form a partition P from 
an equivalence relation R, What is the equivalence re¬ 
lation R that results if we use Theorem 2 again to form 
an equivalence relation from PI 
4(1. Suppose we use Theorem 2 to form an equivalence re¬ 
lation R from a partition P. What is the partition F that 
results if we use Theorem 2 again to form a partition 
from A'' 

41. Devise an algorithm to find the smallest equivalence 
relation containing a given relation. 


*42, Let pin) denote the number of different equivalence 
relations on a set with rt elements (and by Theorem 
2 the number of partitions of a set with n elements). 
Show that pin) satisfies the recurrence relation pin) = 
_ \.j)p(n - j - 1) and the initial condi¬ 
tion p(0) = 1. [Note: The numbers pi n) are called Bell 
numbers after the American mathematician E. T, Bell,] 
43. Use Exercise 42 to find the number of different equiv¬ 
alence relations on a set with n elements where n is a 
positive integer not exceeding 10. 


64 _ 

Partial Orderings 


INTRODUCTION 

We often use relations to order some or all of the elements of sets. For instance, we order 
wards using the relation containing pairs of words (x y) where x comes before y in the 
dictionary. We schedule projects using the relation consisting of pairs (.r, y) where x 
and y are tasks in a project such that x must be completed before y begins. We order 
the set of integers using the relation containing the pairs (x y) where x is less than y. 
When we add all of the pairs of the form (x a) to these relations, we obtain a relation 
that is reflexive, antisymmetric, and transitive. 'These arc properties that characterize 
relations used to order the elements of sets using their relative size. 


DEFINITION 1. A idatdpa if c*n a set S emckting w penial 

order if ft is' reflexive, antisymmet^ AsatS a pmmi 

ordering R is called Apartielly onkred set, nr poset, sad is denoted by (S, R). 


EXAMPLE 1 Show that the "greater than or equal” relation (-’! is a partial ordering on the set of 

integers. 

Solution: Since a a a for every integer a, > is reflexive. If a a h and b '<£ a, then 
a = b. Hence, a is antisymmetric. Finally, 2 is transitive since a > b and b > c 
imply that a £ c. It follows that > is a partial ordering on the set of integers and 
(Z, ) if, a poset, ■ 

EXAMPLE 2 The divisibility relation | is a partial ordering on the set of positive integers, since it is re¬ 

flexive, antisymmetric, and transitive, as was shown in Section 6.1. We see that (Z + ,') 
is a poset. (Z 1 denotes the set of positive integers.) ■ 


EXAM PEE 3 Show that the inclusion relation C is a partial ordering on the power set of a set 5. 

Solution: Since A C A whenever A is a subset (if S, C is reflexive. It is antisymmetric 
since A C B and B C A imply that A ~ B. Finally, C is transitive, since .4 C B and 
B C C imply that AcC. Hence, C is a partial ordering on P($), and (P(5), C) is a 
poset. ■ 
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EXAMPLE 4 


EXAMPLE 5 


EXAMPLE 6 


EXAMPLE 7 


In a poset the notation a <. b denotes that (a, b } t R This notation is used because 
the “less than or equal to” relation is a paradigm for a partial ordering. (Note that the 
symbol « is used to denote the relation in my poset, not just the “less than or equals" 
relation.) The notation a < b denotes that a =? b. but a ¥= h. Also, we say "a is less than 
b” or “b is greater than a” if a < b. 

When a and b are elements of the poset (S. it is not necessary' that either a « b 
or b < a. For instance, in (P( Z), C), {], 2} is not related to {1,3}, and vice versa, since 
neither set is contained within the other. Similarly, in (Z, ), 2 is not related to 3 and 3 
is not related to 2, since 2 / 3 and 3 62. This leads to the following definition. 

DEFINITION 2. The elements a and b of a poset (5, =S) are called comparable, if 
either a < h or b ^ a. When a and b are elements of S such that neither a < b nor 
b A a, a and b are called incomparable. 


In the poset (Z \ j). are the integers 3 and 9 comparable'"’ Are 5 and 7 comparable 1 .' 

Solution: The integers 3 and 9 are comparable, since 3.9. The integers 5 and 7 are 
incomparable, because 5 / 7 and 7 } 5. ■ 


The adjective “partial” is used to describe partial orderings since pairs of dements 
may be incomparable. When every two elements in the set arc comparable, the relation 
is called a total ordering. 

DEFINITION 3. If (5, <) is a poset and every two elements of S are comparable, 

S is called a totally ordered or linearly ordered set, and w is called a total order or 
a linear order. A totally ordered set is also called a chain. 


T he poset (Z, s) is totally ordered, since a h or b ; a whenever a and b are inte¬ 
gers, ■ 


The poset (Z*,,) is no! totally ordered since i! contains elements that are incomparable, 
such as 5 and 7, ■ 


In Chapter 3 we noted that (Z : , sr ) is well-ordered, where ^ is the usuai “less 
than or equals” relation. We now define well-ordered sets. 


DEFINITION 4. (S, <} is a well-ordered set if it is a poset such that < is a total 
ordering and such that every nonempty subset of S has a least element. 


The set of ordered pairs of positive integers, Z + x Z + , with (a h a-,) < {b\. Ic) if «, < 
b ], or if a\ = b i and a-, b : (the lexicographic ordering), is a well-ordered set. The 
verification of this is left as an exercise at the end of this section. The set Z, with the 
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EXAMPLE 8 


EXAMPLE 9 


usual s ordering, is not well-ordered since the set of negative integers, which is a 
subset of Z, has no least element. ■ 

LEXICOGRAPHIC ORDER 

The words in a dictionary are listed in alphabetic, or lexicographic, order, which is 
based on the ordering of the letters in the alphabet. This is a special case of an ordering 
of strings cm a set constructed from a partial ordering on the set. We will show' how this 
construction works in any poset. 

First, we will show how to construct a partial ordering on the Cartesian product of 
two posets, (A i, < i ) and (A a, <2 )• The lexicographic ordering < on A1 x A 2 i s defined 
by specifying that one pair is less than a second pair if the first entry of the first pair is 
less than (in Ai) the first entry of the second pair, or if the first entries are equal, but 
the second entry of this pair is less than (in A;) the second entry of the second pair. In 
other words, (tq, a 2 ) is less than (fq, frj), that is, 

(fli.aj) < {b u bil 

either if a; <, b\ or if both ci) - b\ and < 2 b%. 

We obtain a partial ordering « by adding equality to the ordering -< on A x B. The 
verification of this is left as an exercise. 


Determine whether {3,5) < (4, 8 ), whether (3, 8 ) < (4,5), and whether {4,9) < (4,11) 
in the poset (Z X Z, O. where < is the lexicographic ordering constructed from the 
usual £ relation on Z. 

Solution: Since 3 < 4, it follows that (3, 5) < (4, 8) and that (3, 8) < (4, 5). We have 
(4,9) < (4, 11 ), since the first entries of (4,9) and (4, 11) are the same, but 9 < 11 . ■ 


In Figure ! the set of ordered pairs in Z + x Z + that are less than (3,4) are high¬ 
lighted, 

A lexicographic ordering can be defined on the Cartesian product of n posets 

(Aj. ! >, (A 2 , ^ 2 ).(A,„ <„). Define the partial ordering < on A\ x A 2 x *■ ■ x A„ 

by 

(at.fl2. a„) < ib h b 2 . b a ) 

if ei] <1 h|, or ii there is an integer i > Osuch that U] - b 1 , _ a, - h,, and cij 4 1 <j+j 

h i+i . In other words, one /Muple is less than a second n-tuple if the entry of the first 
n-tuple in the first position where the two n-tuples disagree is less than the entry in that 
position in the second //-tuple. 


Note that (1,2, 3, 5) < (1, 2,4,3), since the entries in the first two positions of these 
4-tuples agree, but in the third position the entry' in the first 4-tuple, 3, is less than that 
in the second 4-tuple, 4. (Here the ordering on 4-tuples is the lexicographic ordering 
that comes from the usual "less than or equals" relation on the set of integers.) ■ 


We can now define lexicographic ordering of strings. Consider the strings 
and bjhi-' bn on a partially ordered set S. Suppose these strings are 
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EXAMPLE lit 


• 

• 

* 

m 

* 

• 

• 

= 1.7) 

! 2. ~ i 

(3,7) 

\4J) 

1.5.7) 

(6.7) 

(7.7) 

* 

• 

• 

• 

• 

• 

• ■ 

Uri 

■ 2, to 


(4, to 

(5,6) 

(6,6) 

(7,6) 

• 

• 

* 

* 

• 

• 

• ■ 

l.to 

t2.M 

(3.3) 

(4,5) 

(5.5) 

(6,5) 

(7.5) 

• 

• 

* 

* 

• 

• 

* - 

1.4) 

12.41 

(3,4) 

(4,4) 

(5,4) 

(6,41 

(7,4) 

* 

* 

* 

• 

* 

• 

* ■ 

U) 

i 2. A ■ 

■3.3 > 

(4,3) 

l-\3) 

(6.3) 

(7,3) 

• 

• 

* 

* 

# 

4 

• - 

1.21 

12,2) 

i3,:i 

(4,2) 

(5,2) 

(6,2) 

(7,2) 

• 

• 

• 

• 

* 

• 

• • 

1.11 


*?..[> 

(4.1) 

(5. 1) 

(0. 1 > 

(7.1) 


FIGURE 1 The Ordered Pairs Less Than [3,4) in Lexicographic Order. 


not equal. Let t be the minimum of m and n. The definition of lexicographic ordering is 
that (he string a\iii - ■ ■ a m is less than b\ bt ■ ■ • b n if and only if 

(ai, aj. a t ) < (b j, bs,..., b r ), or 

(a[, aj . a/) = (t>i, fa . b f ) and m < n, 

where < in this inequality represents the lexicographic ordering of S'. In other words, 
to determine the ordering of two different strings, the longer string is truncated to the 
length of the shorter string, namely, to ( = minim, n) terms. Then the (-tuples made 
up of the iirst t term s of each string are compared using the lexicographic ordering on 
S’. One string is less than another string if the r-tuplc corresponding to the first string 
is Jess man the /-tuple of the second string, or if these two /-tuples are the same, but 
the second string is longer. The verification that this is a partial ordering is left as an 
exercise for the reader. 


Consider the set of strings of lowercase English letters. Using the ordering of letters in 
the alphabet, a lexicographic ordering on the set of strings can be constructed, A string 
is less than a second string if the letter in the Iirst string in the first position where the 
strings differ comes before the letter in the second string in this position, or if the first 
string and the second string agree in all positions, but the second string has more letters. 
This ordering is the same as that used in dictionaries. For example, 

discreet < discrete, 

since these strings differ first in the seventh position, and e < f. Also, 
discreet < discreetness , 

since the first eight letters agree, but the second string is longer. Furthermore. 

discrete < discretion, 
since 

discrete ■< disc ret i. ■ 
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FIGURE 2 Constructing the Hasse Diagram for ({1,2, 3,4}, s£). 


HASSE DIAGRAMS 


Many edges in the directed graph for a finite poset do not have to be shown since 
they must be present. For instance, consider the directed graph for the partial order¬ 
ing {(a, b)\a £ 6} on the set {1,2, 3,4), shown in Figure 2(a). Since this relation is a 
partial ordering, it is reflexive, and its directed graph has loops at all vertices. Conse¬ 
quently, we do not have to show' these loops since they must be present; in Figure 2(b) 
loops are not shown. Because a partial ordering is transitive, we do not have to show 
those edges that must be present because of transitivity. For example, in Figure 2(c) the 
edges (1,3), (1, 4), and (2,4) are not shown since they must be present. If we assume 
that all edges are pointed “upward” (as they are drawn in the figure), w'e do not have to 
show the directions of the edges: Figure 2(c) does not show directions. 

In general, we can represent a partial ordering on a finite set using the following 
procedure. Start with the directed graph for this relation. Because a partial ordering is 
reflexive, a loop is present at every 1 vertex. Remove these loops. Remove all edges that 
must be present because of the transitivity, since they must be present since a partial 
ordering is transitive. For instance, if (a, b) and (b, c) are in the partial ordering, remove 
the edge (a, c), since it must be present also. Furthermore, if (e, d) is also in the partial 
ordering, remove the edge (a, d), since is must be present also. Finally, arrange each 


Helmut Hassr i 1X98—1979), He I mu l li, o was hem m Kassel. He served in the German navy after 
yt'(>h high school. He began his university studies at Gottingen University in 191S, moving in 1920 to Marburg 
University to study under she number theorist Kurt Hcnsel. During this time, Hasse made fundamental 
contributions to algebraic number theory. He became 1 'V n o "s successor at Marburg, later becoming director 
n(" the famous mathematical institute at Gottingen tn 1934, and took a position at Hamburg University in 
1950- Hasse served for 50 years as an editor of Cmlte’x Journal, a famous German mathematics periodical, 
raking over the job of chief editor in 1936 when the Nazis forced Hensello resign. During World Wart t Hasse 
worked tin applied mathematics research for the Gentian navy. He was noted for the clarity and personal 
style of his lectures and was devoted both to number theory and In his students. (Hasse has been controversial 
lor connections with the Nazi party. Investigations have shown he was a strong German nationalist but not 
an ardent Nazi.) 
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edge so that its initial vertex is below its terminal vertex (as it is drawn on paper). 
Remove all the arrows on the directed edges, since ail edges point “upward" toward 
their terminal vertex. (The edges left correspond to pairs in the covering relation of the 
poset. Sec the preamble to Exercise 20.) 

These steps are well-defined, and only a finite number of steps need to be car¬ 
ried out for a finite poset. When all the steps have been taken, the resulting diagram 
contains sufficient information to find the partial ordering. This diagram is called 
a Masse diagram, named after the twentieth century German mathematician Helmut 
Hasse. 


EXAMPLE II Draw the Hasse diagram representing (he partial ordering l(et,b) \ a divides h} on 

{1,2,3,4,6,8,12}, 

Solution: Begin with the digraph for this partial order, as shown in Figure 3(a). Remove 
all loops, as shown in Figure 3(b). Then delete all the edges implied by the transitive 
property. These are (1, 4), (1, 6), (1, 8), (1, 12), (2, 8), (2, 12), and (3, 12). Arrange all 
edges to point upward, and delete all arrows to obtain the Hasse diagram. The resulting 
Hasse diagram is shown in Figure 3(c). 9 


EXAMI’LE 12 Draw the Hasse diagram for the partial ordering {(.4, B) \ A C B] on the power set P(Si 

where S = {a, b. c}. 

Solution: The Hasse diagram for this partial ordering is obtained from the associ¬ 
ated digraph by deleting all the loops and all the edges that occur from transitivity, 
namely, (0, {a, *}). (0, {a, c}). |0, {b, 4), (0, {a b, c}), ({a}, {a, b, c}), ft*}, {a, b. c}), and 
({f}, {a, b, c’}). Finally ail edges point upward, and arrow's are deleted. The resulting 
Hasse diagram is illustrated in Figure 4. ■ 
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EXAMPLE 1.1 


EXAMPLE 14 


1 tJ. h. c[ 




FIGURE 4 The Has* FIGURE 5 The Hasse Diagram 

Diagram of fP({a. b, r», £). of a Posel. 


MAXIMAL AND MINIMAL ELEMENTS 

Elements of posets that have certain extremal properties are important for many appli¬ 
cations. An dement of a poset ts called maximal if it is not less than any element of the 
poset. That is, a is maximal in the posel (3, <) if there is no h 6 S such that a < b. 
Similarly, an element of a poset is called minimal if it is not greater than any element of 
the poset. That is, a is minimal if there is no element b E S such that b < a. Maximal 
and minimal dements are easy U) spot using a Hasse diagram. They are the “tup” and 
'‘bottom” elements in the diagram. 


Which elements of the poset ({2,4.5,10, 12, 20,25}, |) are maximal, and which are 
minimal 

Solution: The Hasse diagram in Figure 5 for this poset shows that the maximal el¬ 
ements are 12, 20, and 25, and the minimal elements are 2 and 5. As this example 
shows, a poset can have more than one maximal element and more than one minimal 
element. ® 


Sometimes there is an element in a puset that is greater than every other dement. 
Such an element is called the greatest element. That is, a is the greatest element of 
the poset (S, <) if b -1 a for all b E S. The greatest element is unique when it exists 
[see Exercise 32(a) at the end nf this section] Likewise, an element is called the least 
element if it is less than all the other elements in the poset. That is, a is the least element 
of (S. O jf (j < h for all b E S. The least element is unique when it exists [see Exercise 
32(b) at the end of the section]. 


Determine whether the posets represented by each of the Hasse diagrams in Figure 6 
have a greatest element and a least element. 

Solution: The least element of the poset with Hasse diagram (a) is a. This poset has no 
greatest element. The poset with Hasse diagram (b) has neither a least nor a greatest 
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EXAMPLE 15 


EXAMPLE 16 


EXAMPLE 17 



d 



a 


(a) (W (cl {di 

FIGURE 6 Hass* Diagrams of Four Post Is. 


dement. The poset with Hasse diagram (c) has no least element, its greatest element is 
d. Tlie poset with Hasse diagram (d) has least element a and greatest dement ci. ■ 


Let 5 be a set. Determine whether there is a greatest dement and a least element in the 
poset (jP(.S'). C). 

Solution: The least element is the empty set, since 0 C T for any subset T of S. The 
set S is the greatest element in this poset, since T C S whenever I is a subset of A. ■ 


Is there a greatest element and a least element in the poset fZ \ J)? 

Solution: The integer 1 is the least element since 1 |n whenever n is a positive integer. 
Since there is no integer that is divisible by all positive integers, there is no greatest 
element. ■ 


Sometimes it is possible to find an element that is greater than all the elements in 
a subset A of a poset ( S , <). If u is an dement of S such that a < u for all elements 
a e A, then u is called an upper bound of ,4. Likewise, there may be an element less 
than all the elements jn A. If [ is an element of S such that / < a for all elements a E 
A, then l is called a lower hound of 4. 


Find the lower and upper bounds of the subsets [a, h, cj, {j, h], and {a. c, d. /} in the 
poset with the Hasse diagram shown in Figure 7. 

Solution: The upper hounds of {a, h, c} are e, f, j, and ft, and its only lower bound is 
a. There are no upper bounds of {j, h}, and its lower bounds are a. b, c, d, e, and f. The 
upper bounds of {a, c, d, f) are f h, and j, and its lower bound is a, ■ 


The element.r is called the least upper bound of the subset 4 if x i s an upper bound 
that is less than every other upper bound of 4. Since there is only one such element, if 
it exists, it makes sense to cali this dement the least upper bound [see Exercise 34(a) 
at the end of this section]. That is, x is the least upper bound of 4 if a < x when¬ 
ever a e A, and x * - whenever z is an upper bound of 4, Similarly, the clement v is 
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EXAMPLE 18 


EXAMPLE 19 



called the greatest lower bound of A if y is a lower bound of A and z ^ v whenever - 
is a lower bound of A, The greatest lower bound of A is unique if it exists [see Exercise 
34( b) at the end of this section]. The greatest lower bound and least upper bound of a 
subset A are denoted by glb(A) and lub(A), respectively. 


Find the greatest lo wer bound and the least upper bound of {it, d. g\, if they exist, in the 
poset shown in Figure 7. 

Solution: The upper bounds of {b. d, g] are g and h. Since g < h , g is the least upper 
bound. The lower bounds of {it, d, g) are a and b. Since a < b,b is the greatest lower 
bound. ■ 


Find the greatest lower bound and the least upper bound of the sets {3,9, 12} and 
{1,2,4, 5, 10} if they exist, in the poset (Z f , |). 

Solution: An integer is a lower bound of {3,9, 12} if 3, 9, and 12 are divisible by this 
integer. The only such integers are 1 and 3. Since 1 [ 3, 3 is the greatest lower bound of 
{3, 9, 12}. The only lower bound for the set {1,2, 4,5, 10} with respect to | is the element 
1. Hence, 1 is the greatest lower bound for {1, 2, 4, 3, 10}. 

An integer is an upper bound for {3,9, 12} if and only if it is divisible by 3, 9, and 
12. The integers with this property are those divisible by the least common multiple of 

3, 9. and 12, which is 36. Hence, 36 is the least upper bound of {3,9,12}. A positive 
integer is an upper bound for the set {], 2,4, 5,10} if and only if it is divisible by 1, 2, 

4, 3, and 10. The integers with this property are those integers divisible by the least 

common multiple of these integers, which is 20. Hence, 20 is the least upper bound of 
{1,2,4,3.10}, ■ 

LATTICES 


A partially ordered set in which ever)' pair of elements has both a least upper bound and 
a greatest lower bound is called a lattice. Lattices have many special properties. Fur¬ 
thermore. lattices are used in many different applications such as models of information 
flow and play an important role in Boolean algebra. 
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EXAMPLE 20 


EXAMPLE 21 


EXAMPLE 22 


EXAMPLE 23 



l : S! I'hj 

i' It JURE 8 Ha-ssc Diagrams of Three Posets. 


Determine whether the posers represented by each of the Has sc diagrams in Figure 8 
are lattices. 

Solution: The posets represented by the Hasse diagrams in (a) and (c) are both lattices 
because in each poset every 1 pair of elements has both a least upper bound and a greatest 
lower bound, as the reader should verify. On the other hand, the poset with the Hasse 
diagram shown in (b) is not a lattice, since the elements h and c have no least upper 
bound. To see this note that each of the elements d, e, and/ is an upper bound, but none 
of these three elements precedes the other two with respect to the ordering of this poset. 


Is the poset (Z 1 . j) a lattice? 

Solution: Let a and b be two positive integers. The least upper bound and greatest lower 
bound of these two integers are the least common multiple and the greatest common 
divisor of these integers, respectively, as the reader should verify. It follows that this 
poset is a lattice. ■ 


Determine whether each of the posets f{ 1,2,3,4.51, |) and ({1, 2,4, 8, 16}, |) is a lattice. 

Solution: Since 2 and 3 have no upper bounds in ({1, 2, 3,4. 5}, |). they certainly do not 
have a least upper bound. Hence, the first poset is not a lattice. 

Every two elements of the second poset have both a least upper bound and a greatest 
lower bound. The least upper bound of two elements in this poset is the larger of the 
elements and the greatest lower bound of two elements is the smaller of the elements, 
as the reader should verify. Hence this second poset is a lattice. ■ 


Detennine whether (P(Si c) is a lattice where S is a set. 

Solution: InM and B be two subsets of S. The least upper bound and the greatest lower 
bound of A and B are A U B and A D B, respectively, as the reader can show. Hence 
(F’(S), C) is a lattice, ■ 
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EXAMPLE 24 

. web 


web 


LEMMA ! 


The Lattice Model of Information Flow In many settings the flow of information 
from one person or computer program to another is restricted via security clearances 
We can use a lattice mode! to represent different information flow policies. For ex¬ 
ample. one common information flow policy is the multilevel security policy used in 
government and military systems. Each piece of information is assigned to a security 
class, and each security class is represented by a pair (A, C) w'here A is an authority- 
level and C is a category. People and computer programs are then allowed access to 
information from a specific restricted set of security classes. 

The typical authority levels used in the U.S. government are unclassified (0), con¬ 
fidential (l). secret (2), and top secret (3). Categories used in security classes are the 
subsets of a set of all compartments relevant to a particular area of interest. Each com¬ 
partment represents a particular subject area. For example, if the set of compartments 
is {spies, moles, double agents}, then there are eight different categories, one for each 
of the eight subsets of the set of compartments, such as {spies, motes}. 

We can order security classes by specifying that (A L , Cj) < (A-o G) if and only if 
A | s: A 2 and G C C 2 . Information is permitted to flow from security class (A [. Cj) 
into security class (A 2 , C?) if and only if (Aj, Cj) (W, Go For example, information 

is permitted to flow from the security class (secret, [spies, moles}) into the security class 
{top secret, {spies, moles, double agents}), whereas information is not allowed to flow 
from the security class [lop secret, {spies, moles}) into either of the security classes 
[secret, [spies, moles, double agents}) or {top secret,{spies}). 

We leave it to the reader (see Exercise 40 at the end of this section) to show that 
the set of all security classes w ith the ordering defined in this example forms a lattice. 


TOPOLOGICAL SORTING 


Suppose that a project is made up of 20 different tasks. Some tasks can be com¬ 
pleted only after others have been finished. How can an order be found for these 
tasks 9 To model this problem we set up a partial order on the set of tasks, so that 
a < b if and only if a and b are tasks where b cannot be started until a has been 
completed. To produce a schedule for the project, we need to produce an order for 
all 20 tasks that is compatible with this partial order. We will show' how this can be 
done. 

We begin with a definition. A total ordering =5 is said to be compatible with the 
partial ordering R if a -% b whenever a R b. Constructing a compatible total ordering 
from a partial ordering is called topological sorting. We will need to use the following 
lemma. 


Every' finite nonempty poset (S, <) has a minimal element. 

Proof: Choose an clement a ( > of S, If «o is not minimal, then there is an element 9] 
with fl| < « ( |. If is not minimal, there is an element a 2 with a 2 < a \. Continue this 
process, so that if a„ is not minimal, there is an element a R + j with a,, (f < a n . Since 
there are only a finite number of elements in the poset, this process must end with a 
minimal dement a„. □ 

The topological sorting algorithm we will describe w'orks for any finite nonempty 
poset. To define a total ordering on the poset (A, <). first choose a minimal dement 
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EXAMPLE 25 


EXAMPLE 26 


Hi; such an element exists by Lemma 1, Next, note that (A - {rq}, <) is also a poset, as 
the reader should verify. If it is nonempty, choose a minimal element a- : of this poset. 
Then remove a? as well, and if there are additional elements left, choose a minimal 
element aj in A - \tii, a?}. Continue this process by choosing aj, h ] to be a minimal 
element in A - {ai, ai >..,, a\, as long as elements remain. 

Since A is a finite set, this process must terminate. The end product is a sequence 
of elements a\, rr> .. a n . The desired total ordering is defined by 

ii i «s a* • < a„. 

This total ordering is compatible with the original partial ordering. To see this, note 
that if h < c in the original partial ordering, c is chosen as the minimal element at a 
phase of the algorithm where h has already been removed, for otherwise c would not 
be a minimal element. Pseudocode for this topological sorting algorithm is shown in 
Algorithm 1. 


ALGORITHM 1 Topological Sorting. 

procedure topological sort (.S’: finite poset) 
k := 1 

while S ¥= 0 
begin 

(3t: - a minimal element of S {such an element exists bv Lemma 1} 
k:= k- 1 

end {ri i . cji. a r _ is a compatible total ordering of A} 


Find a compatible total ordering for the poset ({1, 2,4,5, 12,20}, \), 

Solution: The first step is to choose a mini mal element. This must be 1, since it is 
the only minimal element. Next, select a minimal dement of ({2,4, 5, 12, 20},!). There 
are two minimal elements in this poset, namely, 2 and 5. We select 5. The remaining 
elements are {2,4, 12, 20}. The only minimal element at this stage is 2. Next, 4 is chosen 
since it is the only minimal element of ({4,12, 20}, |). Since both 12 and 20 are minimal 
elements of ({12, 20}, |), either can be chosen next. We select 20, which leaves 12 as 
the last element left. This produces the total ordering 

1<5<2<4<20< 12. 

The steps used by this sorting algorithm are displayed in Figure 9, ■ 


Topological sorting has an application tt> the scheduling of projects. 


A development project at a computer company requires the completion of seven tasks. 
Some of these tasks can be started only after other tasks are finished. A partial ordering 
on tasks is set up by considering task X < task Y if task Y cannot be started until task X 
has been completed. The Hasse diagram for the seven tasks, with respect to this partial 
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FIGURE 4 .4 Topological Sort of ({1, 2,4.5.12, 20}, |). 


ordering, is shown in Figure 10. Find an order in which these tasks can be earned out 
to complete the project. 

Solution: An ordering of the seven tasks can be obtained by performing a topological 
sort. The steps of a sort are illustrated in Figure 1L The result of this sort, A < C < 
R -- £ < F < D < (I, gives one possible order for the tasks. ■ 



\ 

/ \ 

C E 


FIGURE It) The Hasse Diagram for Seven Tasks. 
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42 N tl Kdatinns 


Ex erases 


L Which of me following art* onsets? 

a) 1 Z -I b) (Z, —-1 ci fZ, ^.} d) (Z, 

2, Dtrtvfnniic whether the tvhiti£m> represented by the fol- 
!i*« i nf, .'«i‘ une ntatnccs are partial ord e r r . 

‘ t tt f: l 0 0 f 

in ; o| t» o i (): 

It) o i; 

I i o [ u | 


In F-.xeiv lsi’.s 4 5 determine whether the relation with the 
directed pi apti shown is a partial order. 


j r 



fi. Let to. ' he a poset, Show that ( S, R 1 ) is also a poset, 
wlieie K 1 ;s the inverse of R, The poset fi, ft" 1 ) is 
eailed die dual of (.S'. /?'■ 

7. Kind ‘he duals m the ndlowing posetv 
a' t-|’A !, 21, - 1 
h i (Z, . 

e! \f\Z ; 
di a . i 


8, Which of die lolk'wrrsg pairs or elements are compara¬ 
ble in tin- pusel i ' 

nl o. i5 lit 6, l! el X, lb (I) 7, 7 

9, Kind pvi' irit'ornparaWt* elements in the following 

p’SfK. 

a! i / ! i{o. i. ; 
hi t{t 2-i. o, xt, ,i 

10. I ,et S I. 2. d. 1} With respect to the lexicographic 
order based on the usual "less than" relation, 
a I find ;;L pair; in S' e Das pun i'2, 4), 

hi iittd a i pairs in i S ga'mer than (3. 1 j. 

'■'! dr os' ' i L'.ssc tiiue iLtoi at the posel 


i!. K:inl he Lxiengr.u'hk' ordering of the following 
M-lliplrv 

al (!. 1, 2-., L 2 ! i 

b) tit !." p>, L.L 2 i 

cl iff) t.l.t ! i. to, i, I. Ll); 


12. Find the lexicographic ordering of the following strings 
of lowercase English letters: 

a I quack, quick, quicksilver, quicksand, qimeking 
b: open, opener, opera , operand, opened 
cl aw, zero, zoom, zvology, Tjtwlogical 

13. Find the lexicographic ordering of the bit strings 0.01. 
11.001.010, Oil, 0001, and OKI) based on the ordering 
0 < I. 

14. Draw the Hasse titagrain for the “greater than or 
equals” relation on {(), 1, 2, 3,4, 5J-. 

15. Draw 1 the Basse diagram for divisibility or the set 

a) { 1.2,3.4,5,6.1,8}. b) {1.2,3,5.7, 11,13}. 
c) {1,2, 3.6.12.24,36.48}. d) {I.2.4.H. 16,32.64}. 

16. Draw the Hasse diagram for inclusion on the set P(S. i 
where S = {a, b, c, d). 


In Exercises 17—19 list all ordered pairs in the partial order¬ 
ing with the accompanying Hasse diagram. 



Let (5, <-.) be a poset. We say that an element y € S covers 
an element .v 6 S if x < y and there is no element ; E S 
such that x < ; y. The set of pairs U, v) such that v cov¬ 
ers a is called the covering relation of (5. =%). 

20. What is the covering re i at ion of the partial ordering 
{is, b\ | a divides b] on {1,2,3,4. 6, 12}? 

21. What is the covering relation of the partial ordering 
i.(A B) | A C B\ on the power set of S where S - 
[a, b. c|? 

22. Show I ha 5 the pair (x. y) belongs to the covering rela¬ 
tion of the finite po«t IS, if and only if x is lower 
than f and there is an edge joining x and y in the Hasse 
diagram of this poset. 

23. Show that a finite poset can he reconstructed from its 
covering relation. 

24. Answer the fallowing questions for the partial order 
represented hy the following Hasse diagram. 
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c) Is there a greatest element? 

d) Is there a least element? 

t> Find all upper bounds of {{2|, {4}}, 

f) Find the least upper bound of {{2} {4}}, if it 
exists. 

g) Find all lower bounds of {{I, .1, 4}, {2, 3.4}}. 

h) Find the greatest lower bound of {{1,3.4}, [2. 3. 4ft. 
if it exists. 

28. Give aposet that has 

a) a minimal element but no maximal element. 

b) a maximal element but no minimal element. 

e) neither a maximal nor a minimal dement 

29. Show that lexicographic order is a partial ordering on 
the Cartesian product of two posets. 

30. Show that lexicographic order is a partial ordering on 
the set of strings from a poset. 

31. Suppose that (S, and IT, *%} are posets. Show that 
f S x T, <) is a poset where ft. n a* Cm. vl if and only if 
,v <i II and I -ii v,. 


a) Find the maximal elements. 

It) Find die minimal elements, 
ft [s there a greatest dement? 
d> is there a least dement? 
e! Find ail upper bounds of {n. h, r}. 

f) First! the least upper hound of \a, b, of, if it exists. 

g) Find all lower bounds of {/. g, A}. 

Ill Find the greatest lower bound of {/, g, ft}, if it 
exists, 

25, Answer the I olio wring question 1 ; concerning the poset 
If’. 5,15.24.45}, t. 
a) Find the maximal dements, 
bl Find the minimal elements. 

C) Is there a greatest dement? 

d) Is there a least element? 

e) Find all upper bounds of {3, 5}. 

fl Find the least upper htnind of {3,5}, if it exists, 
gl Find all lower hounds of {15.45}. 

h) Find the greatest lower bound of {15,45}, if u 
exists. 

26,. Answer the- following questions concerning the poset 

({2.4. k 9. 12. IK 27.36,48,60. ?2}. | J. 

a) Find the maximal elements. 

1*1 Find the minimal elements. 
o,i Is there a greatest dement? 

d) Is there a least dement? 

e) Find all upper bounds of {2, 9}. 

0 Find the least upper bound of {2, 9}, if it exists, 
g) Find all lower bounds of {60, 72}. 


32. a) Show that there is exactly one greatest element of a 

poset, if such tin dement exists, 
b) Show that there is exactly one least dement of a 
poset, if such an element exists. 

33. a) Show that there is exactly one maximal dement in 

a poset with a greatest element, 
b) Show that there is exactly one minimal demerit in 
a poset with a least dement. 

34. a) Shaw that the least upper bound of a set in a poset 

is unique if it exists. 

b) Show that the greatest lower bound of a set in a 
poset is unique if it exists. 

35. Determine whether the posets with the fill lowing Her 
diagrams are lattices. 



36. Determine whether the following posets are lattices. 

a) ({1.3. 6,9. 12},!) 

b) til 5 25 125};) 
ct (Z, s.) 

ill (P(H J), where P[$) is die power set of a set S 

37. Show that every nonempty subset of a lattice has a least 
upper bound and a greatest lower bound. 


hi Find die greatest lower hound of {60, 72}, if it 

e\ i.js 

7. Answer the lollowirg questions concerning the poser 
fiilf- i2f. ,df. ;i.2j. }1,41, {2.4}, {3,4}, {1,3.4}, 
{2,3.4},} ci. 

a) Find die maxima: elements, 
bt F md Fie minimal dements, 


38. Show that rf the poset (.V. A) is a lattice then the dual 
poset (.S’, R 1 ) ts also a lattice. 

39. In a company, the lattice model of information How 
is used, to control sensitive information with security 
classes represented by ordered pairs (A.C). Here A 
is an authority level which may be nonproprietary 
tO), proprietary (1), restricted (2), nr registered [3). 
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A category t' is u subset of the set of all projects 
iCfuvltih. Imfwlu. Puma). (Names of animals areoften 
used as Celtic names for projects in companies,) 
a) Is inlormation permitted to flow from (Proprietory, 
{Cheetah, Pumti }) into (Restricted, [Puma})? 
bl Is information permitted to flow from {Restricted, 
{Vtwmh}) into [Registered, [Cheetah- hnpala })? 

c) Into which classes is information from (Pntpri- 
cftHT, [Cheetah, Puma}) permitted to flow? 

d) From which classes is information permit¬ 
ted to flow into the security class [Restricted, 
ympula. Pumti})? 

40. Show that the set .S' of security classes (A O is a lattice, 
where A is a positive integer representing an author¬ 
ity class and C is n subset of a finite set of compart¬ 
ments, with i.4|. fj) s;; {Aj, fVi if and only if Ai Aj 

and tj C C : . [Hint: First show that t.V, <) is a 
poset anti then show that the least upper bound and 
greatest lower bound of (Ai.Cjj and i.A 2 ,<_v are 
tmax(A|. Ac), Cj C CV) and lmin(Ai, A : ). C\ ft Cal, 
respectively,! 

*41. Show that the set of all partitions of a set S with the re¬ 
lation ,P. -s Pj if the pttrtition P i is a refinement of the 
partition P: is a lattice. (See the preamble to Exercise 
27 of Section 6.5.) 

42. Show thitt every totally ordered set is a lattice. 

43. Shi rw that every finite lattice has a least element raid a 
greatest element. 

44. Give :m example of an infinite lattice with 
a) neither a tarsi nor a greatest element, 
bl a (cast but mil a greatest element. 

c) a greatest but not a least element, 
dl hath a least and a greatest dement, 

45. Verily that tZ'x Z',ta is a well-ordered set, 
where rs lexicographic order, as claimed in Exam¬ 
ple 7. 

46. Show that a anile nonempty poset has a maximal ele¬ 
ment. 


47. Find a compatible total order for the poset with the 
(ia.se diagram shown in Exercise 24. 

48. Find a compatible total order for the divisibility relation 
on the set {1,2, 3, 6, 8, 12,24,36}. 

49. Find an order different from that constructed in F.x- 
ample 26 for completing the tasks in the development 
project. 

50. Schedule the tasks needed to build a house, by specify¬ 
ing their order, if the Hasse diagram representing these 
tasks is as shown in the following figure. 



'Roof 


•Framing 

^foundation 


Key Terms and Results 

TERMS 

binary relation from A to ft: a subset of A x B 

relation on A: a binary relation from A to itself (i.e,, a sub¬ 
set of A '■ A) 

.S' o S: coriipuMtc of R and .V 

R inverse relation of R 

R n \ nth power of R 

reflexive: a relation R on ,4 is reflexive if (a, a) <Et R for all 
a e A 

symmetric: a relation A’ on A is symmetric if (b, ;;) e R 
whenever in. b) E R 


antisymmetric: a relation R on A is antisymmetric if a - 
b whenever (a, b) £ R and ( b, u } £ R 
transitive: a relation R on A is transitive if (a, b) E R and 
[b, c) e R implies that (a, ej £ R 
n-ary relation on Ai,Ai, . A-: a subset of A t x A : x 
■ - - X A„ 

relational data model: u model for representing databases 
using n-ary relations 

primary key: a domain of an n-ary relation such that an 
n-tuple is uniquely determined by its value for this 
domain 
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composite key: the Cartesian product of domains of an n- 
jry relation such that an n -tuple is uniquely determined 
by its values in these domains 
projection: a function that produces relations of smaller 
degree from an n-ary relation by deleting fields 
join: a function that combines n-ary relations that agree on 
certain fields 

directed graph or digraph: a set of elements called ver¬ 
tices and ordered pairs of these elements, called edges 
loop: an edge of the form (a, a) 
closure of a relation R with respect to a property P: the 
relation S {if ir cxi sts) that contains R, has property P, and 
is contained within any relation that contains R and has 
property P 

path in a digraph: a sequence of edges (a, jqJAjq, ay), 
.... (Jt n x„. i). [-f, ]. b) such that the terminal vertex 
of each edge is the initial vertex of the succeeding edge 
in the sequence 

circuit (or cycle) in a digraph: a path that begins and ends 
at the same vertex. 

R' (connectivity relation): the relation consisting of those 
ordered pairs {a, b) such that there is a path from a to b 
equivalence relation: a reflexive, symmetric, and transi¬ 
tive relation 

equivalent: if R is an equivalence relation, a is equivalent 
to b if a R b 

[o|r (equivalence class of a with respect to R): the set of 
aii dements of A that are equivalent to a 
|o]„ (congruence class modulo m l: the set of integers 
congruent to a modulo m 

partition of a set 5: a collection of pairwise disjoint 
nonempty subsets that have S as their union 
partial ordering: a relation that is reflexive, antisymmet¬ 
ric, and transitive 

poset f.S, R): a set $ and a partial ordering R on this set 
comparable: the elements a and b in the poset (A. st) are 
comparable if a =? b or b -? a 
incomparable: dements in a poset that are not comparable 
total (or linear) ordering; a partial ordering for which ev¬ 
ery pair of elements are comparable 
totally (or linearly) ordered set: a poset with a total (or 
linear) ordering 

lexicographic order: a partial ordering of Cartesian prod¬ 
ucts or strings (see pages 417-418) 

Hasse diagram: a graphical representation of a poset 


where loops and all edges resulting from the transitive 
property are not shown, and the direction of the edges is 
indicated by the position of the vertices 
maximal element; an element of a poset that is not less 
than any other element of the poset 
minimal dement: an element of a poset that is not greater 
than any other clement of the poset 
least element: an element of a poset less than or equal to 
all other elements in this set 

greatest element: an element of a poset greater than or 
equal to all other elements in this set 
upper bound of a set: an element in a poset greater than 
all other dements in the set 

lower bound of a set: an dement in a poset less than ail 
other elements in the set 

least upper bound of a set: an upper bound of the set that 
is less than all other upper bounds 
greatest lower bound of a set: a lower bound of the set 
that is greater than, all other lower bounds 
lattice: a partially ordered set in which every two elements 
have a greatest lower bound and a least upper bound 
well-ordered set: a poset ( 5 , s£) where =» is a total order 
and every nonempty subset of S has a least element 
compatible total ordering for a partial ordering: a total 
ordering that contains the given partial ordering 
topological sort: the construction of a total ordering com¬ 
patible with a given partial ordering 

RESULTS 

The reflexive closure of a relation R on the set A equals 
R U A, where A = {(g, a) \ a E A}. 

The symmetric closure of a relation A’ on the set A equals 
R U R where R' 1 2 = {(A, a) | («, b) £ A}. 

The transitive closure of a relation equals the connectivity 
relation formed from this relation. 

WarshalPs algorithm for finding the transitive closure of a 
relation (see pages 403 406). 

Let R be an equivalence relation. Then the following three 
statements are equivalent: (I) a A A; (2)fa] R n[b] ff r 0; 

(3) W* = 16]*- 

The equivalence classes of an equivalence relation on a set 
,4 form a partition of A, Conversely, an equivalence re¬ 
lation can be constructed from any partition so that the 
equivalence classes are the subsets in the partition. 

The topological sorting algorithm (sec pages 425-427). 


Review Questions 


1. a) What is a relation on a set? 

b) How many relations are there on a set with « ele 
men is? 

2. a) What is a reflexive relation? 
b) What is a symmetric relation'? 


c) What is an anti symmetric relation? 
d> What is a transitive relation? 

3. Give an example of u relation on the set {1,2,3. 4) 
that is 

a) reflexive, symmetric, and not transitive. 
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b) tun reflexive, symmetric, and transitive, 
el reflexive, antisymmetric, and not transitive. 

d) reflexive, symmetric, and transitive. 

e) reflexive, antisymmetric, and transitive, 

4. a) How many reflexive relations are there on a set with 

n elements? 

b) How many symmetric relations are there on a set 
with n elements? 

cl How many antisymmetric relations are there on a 
set with n elements? 

5. a) Explain how an n-ary relation can be used to repre¬ 

sent information about students at a university. 

b) How ran the 3-ary relation containing names of stu¬ 
dents, their addresses, telephone numbers, majors, 
and grade point averages be used to form a 3-ary 
relation containing the names of students, their ma¬ 
jors. and their grade point averages? 

c) Hnw can the 4-ary relation containing names of 
students, their addresses, telephone numbers, and 
majors and the 4-ary relation containing names of 
students, their student numbers, majors, and num¬ 
bers ot credit hours, he combined into a single n-ary 
relation? 

6. a) Explain how to use a zero-one matrix to represent 

a relation on a finite set. 

b) Explain how to use the zero-one matrix represent¬ 
ing a relation to determine whether the relation is 
reflexive, symmetric, and/or antisymmetric. 

7. a) Explain how to use a directed graph to represent a 

relation oil a finite set. 

b'l Explain how to use the directed graph representing 
a relation to determine whether a relation is re fie x- 
no. symmetric, and/or antisymmetric, 

8. ai Define the reflexive closure and the symmetric 

closure of a relation, 

hi Hovt can you construct the reflexive closure of a 
relation'' 

el J low can you construct the symmetric closure of a 
rdatiofi? 

d) Isnd (lie reflexive closure and the symmetric clo¬ 
sure of the relation {(I, 2), (2. 3), (2,4), (3, I.)} on the 
set {I, 2, 3.4}. 

7. a I Define the transitive closure of a relation. 

b) Can the transitive closure of a relation be obtained 
by including all pairs (a, c) such that (a, b) and 
1 h. cl belong to the relation? 

c) Describe two algorithms for finding the transitive 


closure of a relation. 

d) Find the transitive closure of the relation 

{(U ), (1,3), (2.1), (2,3), (2,4), (3,2), (3.4), (4,1)}, 

10. a) Define an equivalence relation, 

b) Which relations on the set {a, b, c,d\ are equiva¬ 
lence relations and contain {a, b) and (h, d)' } 

11. a) Show r that congruence modulo m is an equivalence 

relation whenever m is a positi ve integer, 
b) Show that the relation {(a, b) \ a zb (mod 7|} is 
an equivalence relation on the set of integers, 

12. a) What are the equi valence classes of an equivalence 

relation? 

b) What are the equivalence classes of the congruent 
modulo 3 relation ? 

c) What are the equivalence classes of the equivalence 
relation in Question 11(b)? 

13. Explain the relationship between equivalence relations 
on a set and partitions of this set, 

14. a) Define a partial ordering. 

b) Show that the divisibility relation on the set of pos¬ 
itive integers is a partial order. 

15. Explain how partial orderings on the sets/l, and Ai can 
be used to define a partial ordering on the set ,4 1 x .Q 

lfi. a) Explain how to construct the Hassc diagram of a 
partial order oil a finite set. 

b) Draw' the Hasse diagram of the divisibility relation 
on the set {2,3.5, 9, 12,15, 18}. 

17. a) Define a maximal element of a poset and the great¬ 

est element of a poset. 

b) Give an example of a poset that has three maximal 
elements, 

c) Give an example o! a pose; with a greatest element. 

18. a) Deline a lattice. 

b) Give an example of a poset with live elements that 
is a lattice and an example of a poset with five ele¬ 
ments that is not a lattice. 

19. a) Show that every finite subset of a lattice has a great¬ 

est lower bound and a least upper bound, 
b) Show that every' lattice with a finite number of ele¬ 
ments has a least element and a greatest element, 

20. a) Define a well-ordered set. 

b) Describe an algorithm for producing a well-ordered 
set from a partially ordered set, 

e) Explain how the algorithm from (b) can be used to 
order the tasks in a project if each task can be done 
only after one or more of the other tasks have been 
completed. 


Supplementary Exercises 


1. Let .3 be the set of all strings of English letters. De¬ 
termine whether the following relations are reflexive, 


irrefiexive. symmetric, antisymmetric, and/or transi¬ 
tive. 
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a’l R-. - {{a, h) | n and b have no letters in common} 

b) R : - j(tt, b) | rt and h are not the same length} 

c) R: = {{a. b'i \ a is longer than A} 

2. (\tnstrufl a relation on the set {a. b, c, d} that is 
a) reflexive, symmetric, hut not transitive. 

I>) irreflesive. symmetric, and transitive, 
c) irreflexive, antisymmetric, and not transitive, 
d; reflexive, neither syrtitnetric nor antisymmetric. 
Kran.silivc. 

c) neither reflexive, irreflexivc. symmetric, antisyni- 
mciric, nor transitive. 

3. Show that the relation ff on Z X Z defined by 
(ti, b) R (c, ti) it and only it u + d - b + t: is an equiv¬ 
alence relation. 

4. Show that a subset of an antisymmetric relation is also 
anti syin metric, 

5. Let R be a reflexive relation on a set A. Show that 

R C R\ 

6. Suppose that Rx and R z are reflexive relations on a set 
A Show that rti 8 R, is irceflexive. 

7. Suppose that Ri and A' ; are reflexive relations on a set 
A, Is Ri n A; also reflexive? Is R\ U R: also reflexive? 

S. Suppose- that R is a symmetric relation on a set A. Is R 
also symmetric? 

9, Lei R\ and R. be symmetric relations, (s R] fl f<; also 
■symmetric? Is ft' U R 2 also symmetric'.' 

10. A relation R is called circular it'd R b and h R t: imply 
dial r R a. .Show- that R is reflexive and circular if and 


only it it is an equivalence relation. 

11. Show that a primary key in an n-ary relation is a pri¬ 
mary key in any projection of this relation that contains 
this key as one of its fields. 

12. Is the primary key in an n-ary relation also a primary 
key jji a larger relation obtained by taking the join of 
this relation with a second relation? 

13. Show that the reflexive closure of the symmetric clo¬ 
sure of a relation is the same as the symmetric closure 
of its reflexive closure. 

14. Let R be the relation on the set of all mathematicians 
that contains the ordered pair la, b i if and only if a and 
b have written a paper together. 

a) Describe the relation R ' 

b) Describe the relation R' 

e) The Erdos number of a mathematician is i if 
this mathematician wrote a paper with the pro¬ 
lific Hungarian mathematician Paul Erdos, it is 2 if 
this mathematician did not write a joint paper with 
Erdos but wrote a joint paper with someone who 
wrote a joint paper with Erdos, and so on (except 
that the Erdos number of F.rdus himself i_s fl.i. Give 
a definition of the Erdos number in terms of paths 
in R, 

*15. a) Give an example to show 1 that the transitive closure 
of the symmetric closure of a relation is not nec¬ 
essarily the same as the symmetric closure of the 
transitive closure of this relation. 


1’aul Erdos (1913-19%), Paul Erdos, born in Budapest. Hungary, was the son of two lugh school ntathd- 
malicv teachers, He was a child prodigy; ill age 3 he could multiply three-digit numbers in his head, and as 
4 he discovered negative numbers on his own, because his mother did not want to expose him to contagious 
diseases, he was mostly home-schooled. At 17 Erdos entered Eotvus University, graduating 4 years later 
with a Ph D. in mathematics. After graduating he spent 4 years at Manchester. England, on a postdoctoral 
fellowship, in 1938 he went to the United States because of the difficult political situation in Hungary, es 
pecislly for Jews. He spent much of his time in the United States, except for 1954 to 1962, when he was, 
banned us part of the paranoia of the McCarthy era. He also spent considerable time m Israel 

Erdus made many significant contributions to combinatorics and to number theory. One at the discos 
erics of which he was most proud is his elementary proof (in the sense that it does not use any complex 
analysts! of the prime, number theorem, which provides art estimate lor the number of primes not exceeding 
a fixed positive integer. He also participated in the modem development of Ramsey theory. 

Erdos traveled extensively throughout the world to work with other mathematicians, visiting confer¬ 
ences. universities, and research laboratories. He almost entirely devoted himself to mathematics, traveling 
from one mathematician to the next, proclaiming "My brain is open." Erdos was the author nr coauthor 
til almost 1500 papers and had almost 300 coauthors. Since he had no permanent home, copies of these 
articles arc kept by Ron Graham, a famous discrete mathematician at AT&T Laboratories, with whom he 
collaborated extensively and who took care of many of his worldly needs. 

Erdos offered rewards, ranging from $10 to $10,000. for the .solution of problems that be found partly ■ 
ularly interesting, with the sive of the reward depending on the difficulty of the problem, He paid out close In 
$4000. Erdos had his own special language, using such terms as "epsilon" (child), "huss” (woman), "slave" 
(man), "captured" (married), “liberated’ - (divorced'), “SupremeFascist" (God), “Sant" (United States), and 
"Joe" (Soviet Union). Although he was curious about many things, he concentrated almost all his cncrjrt on 
mathematical research. He had no hobbies and no full-time JUtf He never married and apparently remained 
celibate. Erdos was extremely generous, donating much of the money he collected from prizes, awards, and 
stipends for scholarships and to worthwhile causes. He traveled extremely lightly and did not like having 
many material possession*. 
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b) Show, however, ihat the transitive closure of the 
symmetric closure of a relation must contain the 
symmetric closure of the transitive closure of this 
relation. 

1(S. a) Let 5 he the set of subroutines of a computer pro- 
grant. Define the relation E by P R Q if subroutine 
P calls subroutine Q during its execution. Describe 
the transitive closure of R. 

b) 1 ’or which subroutines P does fP, Pj belong to the 
transitive closure of/?? 

e) Describe the reflexive closure of the transitive clo¬ 
sure of R. 

17. Suppo.sc dial R and S are relations on a set A with R C 
5 such that the closures of R and S with respect to a 
property P both exist. Show that the closure of R with 
respect to P is a subset of die closure of 5 with respect 
to P. 

18, Show that the symmetric closure of the union of two 
relations is the union of their symmetric closures. 

*19. Devise an algorithm, based on the concept of intenor 
vertices, that finds the length of the longest path be¬ 
tween two vertices in a directed graph, or determines 
that there are arbitrarily long paths between these 
vertices. 

20. Which iif the following arc equivalence relations on the 
set of all people? 

a) ll.v, v) x and v have the same sign of the zodiac} 
hi if.r. vi ' .<■ and y were bom in the same year} 
e) (ix, y) x and y have been in the same city} 

*21. How many different equivalence relations with exactly 
three different equivalence classes are there on a set 
with five elements? 

22. Show that (u, vi I v y E Q}isan equivalence relation 
on the set of real numbers where Q denotes the set of 
rational numbers. What are [J j. [4 J, and [>]? 

2d. Suppose that P, = {A[,Ai_ A m ] and P 2 = {B,_. B 2 , 

.... /?„} are both partitions of the set S. Show that the 
collection of nonempty subsets of the form A ( n B, 
is a partition of S that is a refinement of both P } 
and I’: tsee the preamble to Exercise 27 of Sec¬ 
tion 6.5 !, 

*24. Show that the transitive closure of the symmetric clo¬ 
sure of the reflexive closure of a relation R is the small¬ 
est equivalence relation that contains R. 

25. l et K(.Y) he the set of all relations on a set 5. Define 
the relation re on R(i') by A[ re R 2 if Jfj C R 2 , where 
A’i and A : are relations on 5. Show' that (R(5), re) is a 
poses. 

2b. Let P(.V) be the set til all partitions of the set S. Define 
die relation re on P(S) by A, re P 2 if A, is a refine¬ 
ment of P : (see Exercise 27 of Section 6.5), Show that 
(Pf.ft, re) is a poser 

27. Find tin ordering of the tasks of a software project if the 
Has sc diagram for the tasks of this project is as shown 
on the facing page. 


A subset of a poset such that every two elements of this 
subset are comparable is called a chain. A subset of a poset 
is called an andchain if every two elements of this subset 
are incomparable. 

28. Find all chains in the posets with the Hasse diagrams 
shown in Exercises 17-19 in Section 6.6. 

29. Find all antichains in the posets with the Hasse dia¬ 
grams shown in Exercises 17-19 in Section 6.6. 

30. Find an antiehain with the greatest number of element* 
in the poset with the Hasse diagram of Exercise 24 in 
Section 6 6, 

31. Show that every maximal chain in a linite poset (.S’, re i 
contains u minimal element of S. (A maximal chain is 
a chain that is not a subset of a larger chain.) 

**32. Show that a poset can be partitioned into k chains, 
where k is the largest number of elements in an an¬ 
tichar n in (his poset. 

*33. Show that in any group of m/i-+ 1 people there is either a 
list of m \ I people where a person in the list (except for 
the first person listed) is a descendant of the previous 
person on the list, or there are n + I people such that 
none of these people is a descendant of any of the other 
n people. {Hint: Use Exercise 32.) 

*34. Establish the generalized induction principle: P\.x) 
is true for every element a in a well-ordered set S if 
Pfjtttl is true, where .V(, is the least element of 5 (the 
basis case), and if P(.r) is true for all r < v. then Pi vl 
is true (the inductive step). 

35. Use the generalized induction principle on the well- 
ordered set (Z' U {()} X Z 4 U {0}) (with lexicographic 
ordering) to show that a,, liFI •- |ntn ■( I )/2| - m, where 
uni j - 0 and 

f r I if u 0. 

1 i ■ n if n re- 0. 

A relation R on a set A is a quasi-ordering on .4 if R is 
reflexive and transitive. 

36. i .et. R be the relation on the set of all functions from Z' 
to Z' such that (f.g) belongs to R if and only iff is 
0(g). Show that R is a quasi-ordering, 

37. Let A be a quasi-ordering on a set A. Show that J?n R 1 
is an equivalence relation. 

*38. I ret A be a quasi-ordering and let S be the relation on the 
set of equivalence classes of R fi R 1 such ihat (C, 13) 
belongs to S. where C and D are equivalence classes of 
R. if and only if there arc elements c of C and d of D 
such that (c d) belongs to A. Show that S is a partial 
ordering. 

Let !, be a lattice. Define the meet ( A) and join (yj opera¬ 
tions by x A y = glbf.tr. v] and x y y - lubf.v. v). 

39, Show that the following properties hold for all elements 
x, y, and ; of a lattice L. 
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al i v y A, .r and a \, y - y v a (commutative 
laws) 

It) u-\i )A~ - ,yA(vAy)and{-ry’) , )\/c 5. ,*v : i>’Vc) 

(associative laws) 

c) a .(a v.v) ~ a and ,v v t.vA v) = -v (absorption 
laws) 

«l) t a - .r and .v \/ v - a (idempotent laws) 

4(1, Show that ff'.v and y are dements of a lattice L then 
a V y y if and only if x A y = x. 

A lattice /. is bounded if it has both an upper bound, 
denoted by I, such that r a 1 for all a S. L and a lower 
bound, denoted by (), such that 0 -S x for all .1 E L 

4(. Show that if/, is a bounded lattice with upper bound 1 
and lower bound 0 then the following properties hold 
for aJ! elements ,v E /.. 
a) i \ I ■■ [ hi .r A 1 - i 

t) 1 y 11 ,t il) ,yAO - 0 

42,. Show thiil every finite lattice is bounded. 


A lattice is called distributive if x V Cy A z) - (x y y)A 
(a V z) and a-A( yy-) -- (jrAyJvfjrA;) for all a. y, and 
in /.. 

*43. Give an example of a lattice which is not distributive. 

44. Show that the lattice (P(S) c) where l\S,) is the power 
set of a finite set S is distributive, 

45. Is the lattice (Z + , |) distributive? 

The complement of an element a of a bounded lattice /. 
with upper bound 1 and lower bound (f is an element h such 
that a V b - 1 and a A b = 0, Such a lattice is comple¬ 
mented if every element of the lattice has a complement, 

46. Give an example of a finite lattice where at least one 
element has more than one complement and at least one 
element has no complement. 

47. Show that the lattice ( P(S). C) where P(S) is (ite power 
sel of a finite set 5 is, complemented. 

*48. Show that if f. is a finite distributive lattice, then an 
element of /, has at most one complement. 
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Computer Projects 

V, R!!'K I’ROl. iK.\MS 'AOH HIT fill. EGA !M, INPUT AMI Oi l RIM. 


1. (liven the matrix rep re sen ting a relation on a finite >ei. 
determine whether the rclatinn is reflexive and/or ir- 
reflexive. 

2. Given the matrix representing, a relation or a finite set, 
determine whether the relation is symmetric and/or an¬ 
tisymmetric. 

3. Given the matrix representing a relation on a finite set, 
determine whether the relation is transitive, 

4. Gtven a positive integer «. display all the relations on 
a set with n elements. 

*5. Given a jmsitive integer n, determine the number of 
transitive relations mi is set with >i elements. 

*(i. Given a positive integer n, determine the number of 
equivalence relations on a set with n elements. 

"7, Given a positive integer n. display all the equivalence 
relations on the set of the ri smallest positive integers. 

8. Given an n-ary relation, find the projection of this rela¬ 
tion when specified fields are deleted. 

Give an et-ary relation and an n-ary relation, and a set 
of common fields, find the join of these relations with 


respect to these common holds. 

10. Gi vert the matrix representing a relation on a finite set. 
Sind the matrix representing the reflexive closure of this 
relation. 

11. Given the matrix representing a relation on a finite set. 
find the matrix representing the symmetric closure of 
this relation, 

12. Given the matrix representing a relation on a finite set. 
find (he matrix representing the transitive closure of 
this relation by computing the join of the powers of the 
matrix representing the relation. 

13. Given the matrix representing a relation on a finite 
sei, find the matrix representing the transitive closure 
of this relation using Warshall's algorithm. 

14. Gi sen the matrix representing a relation on a finite set. 
find the mains representing the smallest equivalence 
relation containing this relation. 

15. Given a partial ordering on a finite sci, find a total 
ordering compatible with it using topological 
sorting. 


Computations and Explorations 

USE AOtMl'l TUiUNAl. PROGRAM OR PROGR VMS \ (H' HAVE WJGTTEN in DM 
Tilt' PCM I OWING ENERGISES 


1. Display alt the dilterem relations on a set with four 
elements. 

2. Display all the different reflexive and symmetric rela¬ 
tions on a -.et with six elements, 

3. Display all the reflexive and transitive relations on a set 
with live elements. 

*4. Determine how many transitive relations there tire on 
a set with n elements for all positive integers n with 
n i 7. 

?. hind the transitive closure of a relation of your choice on 
a set with at least 20elements, Either use a relation that 


corresponds to direct links in a particular transportation 
or enniniunicutions network or use a randomly generated 
relation. 

fi. Compute the number of different equivalence relations 
on a set with n elements for all positive integers n not 
exceeding 20. 

7. Display all the equivalence relations on a set with seven 
elements. 

*8. Display all the partial orders cm a set with five cle- 
meti t.s. 

*9. Display ah the lattices on a set with five elements. 


Writing Projects 

KI.M'GV TO THh f-XH.LOWING QUESTIONS WITH ESS Vi'S 

1. Discuss the concept of a fuzzy relation. How are fuzzy 
relations used? 

2. Descrik- the basic principles of relational databases, 
going beyond what was covered in Section 6.2, How 


USING01. ['SIDE COERCES 

widely used are relational databases as compared with 
other types nf databases'.' 

3. Look up the original papers by Warshall and by Roy (in 
Erench) in which they develop algorithms for finding 
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transitive closures. Discuss their approaches. Why do 
you suppose that what we call Warshall’s algorithm was 
discovered independently by more than one person? 

4. Describe: how equivalence classes can be used to define 
the rational numbers as classes of pairs of integers and 
how the basic arithmetic operations on rational num¬ 
ber:- can be defined following this approach. (Sec Ex¬ 
ercise 10 in Section 6,3.1 

5. Explain how Helmut Hesse used what we now call 
Masse diagrams. 

6. Describe some of the mechanisms used to enforce in¬ 
formation How policies in computer operating systems. 


7. Discuss the use of the Program Evaluation and Review 
Technique (PERT) to schedule the tasks of a large com¬ 
plicated project. How widely is PERT used? 

8. Discuss the use of the Critical Path Method (CPM) to 
find the shortest time for the completion of a project. 
How widely is CPM used? 

9. Discuss the concept of duality in a lattice. Explain how 
duality can be used to establish new results. 

10. Explain what is meant by a modular faith?. Describe 
some of the properties of modular lattices and describe 
how modular lattices arise in the study of projective 
geometry. 
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Graphs 



G raph theory is an old subject with many modem applications. Its basic ideas 
were introduced in the eighteenth century by the great Swiss mathematician 
Leonhard Euler. He used graphs to solve the famous Kbmgsberg bridge prob¬ 
lem, which we will discuss in this chapter. 

Graphs are used to solve problems in many fields. For instance, graphs can be 
used to determine whether a circuit can be implemented on a planar circuit board. We 
can distinguish between two chemical compounds with the same molecular formula 
but different structures using graphs. We can determine whether two computers are 
connected by a communications link using graph models of computer networks. Graphs 
with weights assigned to their edges can be used to solve problems such as finding the 
shortest path between two cities in a transportation network. We can also use graphs to 
schedule exams and assign channels to television stations. 


7.1 __ 

Introduction to Graphs 

Graph' are discrete structures consisting of vertices and edges that connect these ver- 
y ( 'gfo trees- There are several different types of graphs that differ with respect to the kind and 
number of edges that can connect a pair of vertices. Problems in almost every conceiv¬ 
able discipline can be solved using graph models. We will give examples to show how 
craphv are used a* models in a variety of areas. Lor instance, we will show how graphs 
are used io represent the competition of different species in an ecological niche, how 
graphs are used to represent who influences whom in an organization, and how graphs 
are used to represent the outcome of tournaments. Later we will show how graphs can 
be used to solve many types of problems, such as computing the number of different 
combinations of (lights between two cities in an airline network, determining whether 
t[ is possible to walk down all the streets in a city without going down a street twice, 
and finding the number of color*, needed to color the regions of a map. 

TYPES OF GRAPHS 


We will introduce the different topes ol graphs bv showing how each can be used to 
model a computer network. Suppose that a network is made up of computers and tele¬ 
phone lines between computers. We can represent the location of each computer by a 
[■uiint and each telephone line In an arc. as shown in Figure 1. 

We make the following observations about the network in figure 1. There is tit 
most one telephone line between two computers in this network, each line operates 
m boil: directions, and no computer has a telephone fine to itself. Consequently this 
network can be modeled using a simple graph, consisting of vertices that reprcscni 
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Detroit 



the computers and undirected edges that represent telephone lines, where each edg ? 
connects two distinct vertices and nn two edges connect the same pair of vertices. 


DEFINITION 1 . A simple graph G = (V,E) consists of V, a nonempty set of 
vertices, and E, a set of unordered pairs of distinct elements of V called edges. 


Sometimes there are multiple telephone lines between computers in a network, This 
is the case when there is heavy traffic between computers, A network with multiple 
lines is displayed in Figure 2. Simple graphs are not sufficient to model such networks. 
Instead, multigraphs are used, which consist of vertices and undirected edges between 
these vertices, with multiple edges between pairs of vertices allowed, Every simple 
graph is also a multigraph. However, not all multigraphs are simple graphs, since in a 
multigraph two or more edges may connect the same pair of vertices. 

We cannot use a pair of vertices to specify an edge of a graph when multiple edges 
are present. This makes the formal definition of multigraphs somewhat complicated. 

DEFINITION 2. A multigraph G = ( V, E ) consists of a set ¥ of vertices, a set 
E of edges, and a function / from E to {{«, v} j u, v € V, u s* v}. The edges e\ and e% 
are called multiple or parallel edges if / ie\ ) - f(ei). 

A computer network may contain a telephone line from a computer to itself (per¬ 
haps for diagnostic purposes). Such a network is shown in Figure 3, We cannot use 
multigraphs to model such networks, since loops, which are edges from a vertex to it¬ 
self. are not allowed in multigraphs. Instead, pseudographs are used. Pseudographs are 
more general than multigraphs, since an edge in a pseudograph may connect a vertex 
with itself. 

To formally define pseudograph we must be able to associate edges to sets contain¬ 
ing just one vertex. 



LAnodes 


FIC. I RK 2 A Computer Network with Multiple Lines. 
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Detroit 



FIGURE 3 A Computer Network with Diagnostic Lines, 


DEFINITION 3. A pseudograph G = (V' E) consists of a set V of vertices, a 
set £ of edges, and a function/ firom E to {{u, v) j h, v £ V"}. An edge is a hop if 
f(e) = {«,«} = {«} for some u € V. 


The reader should note that multiple edges in a pseudograph are associated to the 
same pair of vertices. However, we will say that {«. v} is an edge of a graph G - f V, E) 
if there is at least one edge e with /(e) = {a. v}. We will not distinguish between the 
edge e and the set {«, v} associated to it unless the identity of individual multiple edges 
is important. 

To summarize, pseudograplis are the most genera! type of undirected graphs since 
they may contain loops and multiple edges. Multigraphs are undirected graphs that may 
contain multiple edges but may not have loops. Finally, simple graphs are undirected 
graphs with no multiple edges or loops. 

The telephone lines in a computer network may not operate in both directions. For 
instance, in Figure 4 the host computer in New York can only receive data from other 
computers and cannot send out data. The other telephone lines operate in both directions 
and are represented by pairs of edges in opposite directions. 

We use directed graphs (which were studied in Chapter 6) to model such networks. 
The edges of a directed graph are ordered pairs. Loops, ordered pairs of the same ele¬ 
ment, are allowed, but multiple edges in the same direction between two vertices are 
not. Recall the following definition. 

DEFINITION 4. A directed graph (V, E) consists of a set of vertices V and a set 
of edges E that are ordered pairs of elements of V. 

Finally, multiple lines may be present in the computer network, so that there may 
be several one-way lines to the host in New York from each location, and perhaps mure 
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FIGl'RF. 5 A Computer Network with Multiple One-Way Linas. 


titan one line back to each remote computer from the host. Such a network is shown in 
Figure 5. Directed graphs are not sufficient for modeling such a network, since multiple 
edges are not allowed in these graphs. Instead, directed multigraphs, which may have 
multiple directed edges from a vertex to a second (possibly the same) vertex, are needed. 
The formal definition of a directed rtiultigraph follows, 

DEFTNITTON5. A directed multigraph G ~ (V,E ) consists of a set V’ of vertices, 
a set E of edges, and a function / from E to {(«, v) j «, v E V }. The edges e\ and ej 
are multiple edges if f{e\) = /(es). 


The reader should note that multiple directed edges are associated to the same pair 
of vertices. However, we will say that iu, v) is an edge of G - (V, E) as long as there 
is at least one edge e with /(e) - (rev). We will not make the distinction between 
the edge e and the ordered pair («, v) associated to it unless the identity of individual 
multiple edges is important. 

This terminology tor the various types of graphs makes clear whether the edges of a 
graph arc associated to ordered or unordered pairs, whether multiple edges are allowed, 
and whether loops are allowed. We will use graph to describe graphs with directed 
or undirected edges, with or without loops and multiple edges. We will use the terms 
undirected graph or pseudograph for an undirected graph that may have multiple 
edges anti loops. We will always use the adjective directed when referring lo graphs 
thiU have ordered pairs associated to iheir edges. The definitions of the various types of 
graphs are summarized in Table 1. Because of the relatively modem interest in. graph 
theory', and because it has applications to a wide variety of disciplines, many different 
terminologies of graph theory are commonly used. The reader should determine how 
such terms are being used whenevei they are encountered, Perhaps this terminology 
will become standardized someday. 


T.ABI.K 1 Graph Terminology. 


Type 

Edges 

Multiple Edges Allowed? 

Loops Allowed? 

Simple graph 

Unmisumd 

No 

No 

i Mukigraph 

Undirected 

Yes 

No 

Pseudograph 

Undirected 

Yes 

Yes 

Directed graph 

Directed 

No 

Yes 

j Directed maltigraph 

Directed 

Yes, 

Yes 
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EXAMPLE I 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


GRAPH MODELS 


Graphs are used in a wide variety of models. We w ill present a few graph models from 
diverse fields here. Others will be introduced in subsequent sections of this and the 
following chapters. 


Niche Overlap Graphs in Ecology Graphs are used in many models involving the 
interaction of different species of animals. Fur instance, the competition between 
species in an ecosystem can he modeled using a niche overlap graph. Each species 
is represented by a vertex. An undirected edge connects two vertices if the two 
species represented by these vertices compete (that is, some of the food resources 
the; use are the same). The graph in Figure 6 models the ecosystem of a forest. We 
see from this graph that squirrels and raccoons compete but that crows and shrews 
do not. ■ 


Influence Graphs In studies of group behavior it is observed that certain people can 
influence the thinking of others. A directed graph called an influence graph can be 
used to model this behavior. Each person of the group is represented by a vertex. There 
is a directed edge from vertex a to vertex b when the person represented by vertex 
ti influences the person represented by vertex b. An example of an influence graph 
for members of a group is shown in Figure 7, In the group modeled by this influence 
graph, Deborah can influence Brian, Fred, and Linda, but no one can influence her. 
Also, Yvonne and Brian can influence each other. ■ 


Round-Robin "Iournaments A tournament where each team plays each other team ex¬ 
actly once is called a round-robin tournament. Such tournaments cart he modeled 
using directed graphs where each team is represented by a vertex. Note that tu, h) is an 
edge if team a beats team b. Such a directed graph model is presented in Figure 8. Note 
that Team 1 is undefeated in this tournament, and Team 3 is winless. ■ 


Precedence Graphs and Concurrent Processing Computer programs can be executed 
more rapidly by executing certain statements concurrently. It is important not to 



Linda Brian 



IJcbura/t Hvd Yvnattp 

FIGl RE 7 An Influence Graph. 
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Team 

5 


Team 

4 


FIGURE 8 A Graph Model Of a 
Round-Robin Tournament. 



FIGURE 9 A Precedence Graph. 


execute a stateme nt that req uires results of statements not yet executed. The dependence 
of statements on previous statements can be represented by a directed graph, F.aeh 
statement is represented by a vertex, and there is an edge from one vertex to a second 
vertex if the statement represented by the second vertex cannot be executed before the 
statement represented by the first vertex has been executed. This graph is called 
a precedence graph. A computer program and its graph are displayed in Figure 9. 


For instance, the graph shows 
Si, Si, and S 4 tire executed. 


Exercises 

1. Draw graph models, stating the type of graph used, 
to represent airline routes where every day there are 
four flights Irani Boston to Newark, two (lights from 
Newark to Boston, three flights from Newark to Mi¬ 
ami. two ilights from Miami to Newark, one flight from 
Newark in Detroit, two flights front Detroit to Newark, 
three I lights from Newark to Washington, two (lights 
from Washington to Newark, and one (light from Wash¬ 
ington to Miami, with 

ai an edge between vertices representing cities that 
have a flight between them (in either direction). 

b) an edge between vertices representing cities for 
each fl ight ihat operates between them (in cither di¬ 
rection!. 

c) an edge between vertices representing cities for 
each flight that operates between them (in either di¬ 
rection). plus a loop for a special sightseeing trip 
that takes off and lands in Miami. 

d) an edge from a vertex representing a city where a 
flight starts 10 the vertex representing the city where 
it ends. 

el an edge for each (light from a vertex representing a 
cirv where the flight begins to the vertex represent¬ 
ing the citv where the flight ends. 


that statement S$ cannot be executed before statements 


2. What kind of graph can be used to model a highway 

system between major cities where 

a) there is an edge between the vertices represent¬ 
ing cities if there is an interstate highway between 
them? 

b) there is an edge between the vertices representing 
cities for each interstate highway between them? 

e) there is an edge between the vertices representing 
cities for each interstate highway between them, 
and there is a loop at the vertex representing a city if 
there is an interstate highway that circles this city? 

For Exercises 3-9, determine whether the, graph shown is 
a simple graph, a multigraph (and not a simple graph), a 
pseudograph (and not a multigraph), a directed graph, or a 
directed multigraph (and not a directed graph). 



d c d 
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10. For each undirected graph in Exercises 3-9 that is not 
simple, find a set of edges to remove to make it simple. 

11. The intersection graph of a collection of sets A i, Aj, 

..., A„ is the graph that has a vertex for each of these 
sets and has an edge connecting the vertices represent¬ 
ing two sets if these sets have a nonempty intersec¬ 
tion. Construct the intersection graph of the following 
collections of sets. 

a) 4, - {(12,4,6.8}, A : = {0, 1,2, 3.4}. 

Ai = {1.4 5.7,9}, .4, {5,6. 7.8,9}. 

A f = ( 0 . 1 . 8 . 9 } 

b) 4[ = {....-4, -5, -2. - 1,0}, 

.4- -2, -1,0. I, 2 ,...}, 

A\ = {...,-6. -4, -2,0.2 4.6....}, 

A 4 - {— -5. -3. -1, 1.3,5, 

4 s -• {— 6.-3, 0,3,6,,..} 
e) 4, - lx | x < 0}, 

4 : ■ {.v ! - I < a- < 0}, 


A] - \x ■ 0 < X « I}, 

4 4 - {.t: -1 < x < I}, 

4 5 = }x j x > - 1 }. 

4 fj = R. 

12. Use the niche overlap graph in Figure 6 to determine 
the species that compete with hawks. 

13, Construct a niche overlap graph for six species of birds 
where the hermit thrush competes with the robin and 
with the blue jay, the robin also competes with the 
mockingbird, the mockingbird also competes with the 
blue jay. and the nuthatch competes with the hairy 
woodpecker. 

Id. Who can influence Fred and whom can Fred influence 
in the influence graph in Example 2? 

15. Construct an influence graph for the board members of 
a company if the President can influence the Director 
of Research and Development, the Director of Market¬ 
ing, and the Director of Operations; the Director of Re¬ 
search and Development can influence the Director of 
Operations; the Director of Marketing can influence the 
Director of Operations; and no one can influence, or be 
influenced by, the Chief Financial Officer. 

lb. Which other teams did Team 4 beat and which teams 
beat Team 4 in the round-robin tournament represented 
by the graph in Figure 87 

17, In a round-robin tournament the Tigers beat the Blue 
Jays, the Tigers beat the Cardinals, the Tigers beat the 
Orioles, the Blue Jays beat the CardinaJs, the Blue Jays 
beat the Orioles, and the Cardinals beat the Orioles. 
Model this outcome with a directed graph. 

18, Which statements must be executed before S& is exe¬ 
cuted in the program in Example 47 (Use the prece¬ 
dence graph in Figure 9.) 

19, Construct a precedence graph for the fallowing pro¬ 
gram: 


S 1 : A 

= 0 

S 2 : x 

= A t- 1 

Si : v 

- n 


" y 

5s : a 

= x+ 2 

$6 : y 

= x + z 

S- : z 

- 4 


20. Describe a discrete structure based on a graph that can 
be used to model airline routes and their flight times. 
{Him; Add structure to a directed graph.) 

21. Describe a discrete structure based on a graph that can 
be used to model relationships between pairs of indi¬ 
viduals in a group, where each individual may either 
like, dislike, or be neutral about another individual, 
and the reverse relationship may be different. (Him: 
Add structure to a directed graph. Treat separately the 
edges in opposite directions between vertices repre¬ 
senting two individuals.) 
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7.2 _ 

Graph Terminology 

INTRODUCTION 


We introduce some of the basic vocabulary of graph theory in this section. We will use 
this vocabulary when we solve many different types of problems. One such problem 
involves determining whether a graph can be drawn in the plane so that no two of its 
edges cross. Another example is deciding whether there is a one-to-one correspondence 
between the vertices of two graphs that produces a one-to-one correspondence between 
the edges of the graphs. We will also introduce several important families of graphs 
often used as examples and in models. 


BASIC TERMINOLOGY 


First, we give some terminology that describes the vertices and edges of undirected 
graphs. 

DEFINITION 1. Two vertices u and V in an undirected graph G are called adja¬ 
cent (or neighbors) in G if {«, c} is an edge of G. if e - {«, v}, the edge e is called 
incident with the vertices u and v. The edge e is also said to connect u and v. The 
vertices u and v are called endpoints of the edge {», v}. 


To keep track of how many edges are incident to a vertex, we make the following 
definition. 

DEFINITION 2. The degree of a vertex in an undirected graph is the number of 
edges incident with it, except that a loop at a vertex contributes twice to the degree 
of that vertex. The degree of the vertex v is denoted by deg(v). 


EXAMPLE 1 What arc the degrees of the vertices in the graphs G and H displayed in Figure 1? 

Solution: In G, deg(«) = 2, deg(b) = deg(c) = deg(/) = 4, deg(d) - 1, deg{e) = 
3, and deg(g) = 0. In H. deg(a) = 4, deg(h) - deg(e) - 6, deg(c) = 1, and 
deg(rt) = 5. ■ 



__ G __ _ h 

FIGURE 1 The Undirected Graphs G and H. 
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THEOREM 1 

A vertex of degree 0 is called isolated. It follows that an isolated vertex is not 
adjacent to any vertex. Vertex g in graph G i n Example 1 is isolated. A vertex is pendant 
if and only if it has degree 1. Consequently, a pendant vertex is adjacent to exactly one 
other vertex. Vertex d in graph G in Example 1 is pendant. 

What do we get when we add the degrees of all the vertices of a graph G - ( V, £')? 
Each edge contributes 2 to the sum of the degrees of the vertices since an edge is incident 
with exactly two (po ssibly equal ) vertices. This means that the sum of the degrees of the 
vertices is twice the number of edges. We have the following result, which is sometimes 
called the Handshaking Theorem, because of the analogy between an edge having two 
endpoints and a handshake involving two hands. 

The Handshaking Theorem Let G = (V, E) be an undirected graph with e edges. 

Then 

2 e ~ ^ deg(v). 

»-ev 

(Note that this applies even if multiple edges and loops ate present. ) 

EXAM PI. K 2 

How many edges are there in a graph with 10 vertices each of degree 6? 

Solution: Since the sum of the degrees of the vertices is 6 - 10 = 60, it follows that 
le - 60. Therefore, e - 30. ■ 

THEOREM 2 

Theorem 1 shows that the sum of the degrees of the vertices of an undirected graph 
is even. Th is simple fact has many consequences, one of which is given as Theorem 2. 

An undirected graph has an even number of v ertices of odd degree, 

Proof: Let V\ and IT be the set of vertices of even degree and the set of vertices of 
odd degree, respectively, in an undirected graph G = (V, E). Then 

2e = Vdcg(v) = dcyt i + V dcg(v). 

i-ev i-er. i-evi 

Since deg(y) is even for v t Vj, the first term in the right-hand side of the last equality 
is even. Furthermore, the sum of the two terms on the right-hand side of the last equality 
is even, since this sum is 2e, Hence, the second term in the sum is also even. Since ail 
the terms in this sum are odd, there must be an even number of such terms, Thus, there 
are an even number of vertices of odd degree. □ 

There is also some useful terminology for graphs with directed edges. 

DEFINITION 3, When («, v) is an edge of the graph G with directed edges, u is 
said to be adjacent to v and v is said to be adjacent firm u. The vertex «is catted the 
initial vertex of (a, v), and v is called the terminal or end vertex of (a, v). The initial 
vertex and terminal vertex of a loop are the same. 
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EXAMPLE 3 


THEOREM J 



FIGURE 2 The Directed Graph G. 


Since the edges in graphs with directed edges are ordered pairs, the definition of 
the degree of a vertex can be refined to reflect the number of edges with this vertex as 
the initial vertex and as the terminal vertex. 


DEFINITION 4. In a graph with directed edges th c in-ikgree of a wiS|» v, de¬ 
noted by deg~(v), is the number of erfg^s with v as tfefr btarmhud vertex. Use <mt- 
degree of v, denoted by deg + (v), is the number of edges v as tMif initial vertex 

(Note that a loop at a vertex contributes 1 to both the in-efc^rree tffld tbe oat-degree 
of this vertex.) 


Find the in-degree and out-degree of each vertex in the graph G with directed edges 
shown in Figure 2. 

Solution: The in-degrees in G are: deg” (a) = 2, deg '(b) = 2, deg (e) = 3, 
deg" (d) = 2, deg (e) - 3, and deg (/) = 0. The out-degrees are: deg + (tt) - 4. 
deg + (h) = I.dcg'(c) - 2,deg ’'(d) = 2,deg' (e) = 3. anddeg'f/) ■= 0. ■ 


Since each edge has an initial vertex and a terminal vertex, the sum of the in¬ 
degrees and the sum of the out-degrees of ail vertices in a graph with directed edges 
are the same. Both of these sums are the number of edges in the graph. This result is 
stated as the following theorem. 

Let G = (K E) be a graph with directed edges. Then 
Vdeg‘(v) = ^deg + (v) = (4 

vEV VSV 


There are many properties of a graph with directed edges that do not depend nn the 
direction of its edges. Consequently, it is often useful to ignore these directions. The 
undirected graph that results from ignoring directions of edges is called the underlying 
undirected graph. A graph with directed edges and its underlying undirected graph 
have the same number of edges. 


SOME SPECIAL SIMPLE GRAPHS 


We will now introduce several classes of simple graphs. These graphs are often used 
as examples and arise in many applications 
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EXAMPLE 4 Complete Graphs The complete Staph on n vertices, denoted by K n , is the simple 

graph that contains exactly one edge between each pair of distinct vertices. The graphs 
K„, for n = 1, 2,3,4, 5, 6 , are displayed in Figure 3. ■ 


EXAMPLE 5 Cycles The cycle C„, n > 3, consists of n vertices v\, Yj .v„ and edges {v ]( 

{i-' 2 , vi},..., {v ri _ ]p v„}, and {v„, V|}. The cycles Cj, C 4 , Cs» and Cg are displayed in 
Figure 4. ■ 



C;„ C, tV C, 


FIGURE 4 The Cycles Cj, C 4 , C$, and C t ,■ 


EXAMPLE 6 Wheels We obtain the wheel W n when we add an additional vertex to the cycle C n , 

for/; > 3, and connect this new vertex to each of the n vertices in C„, by new' edges. 
The wheels HC, VU 4 , VPs, and iVV, tire displayed in Figure 5. ■ 

EXAMPLE 7 /i-Cubes The n-cube, denoted by Q„, is the graph that has vertices representing the 2" 

bit strings of length n. Two vertices are adjacent if and only if the bit strings that they 
represent differ in exactly one bit position. The graphs Q\ , Qi, and Q\ are displayed in 
Figure 6 . ■ 



HGURE 5 The Wheels W M W 4 , Wj, and W*. 
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EXAMPLE 8 


EXAMPLE 9 



y, c> 2 Qi 

FIGURE 6 The n-cube Q„ for n = 1,2, and 3. 


BIPARTITE GRAPHS 


Sometimes a graph has the property that its vertex set can be divided into two disjoint 
subsets such that each edge connects a vertex in one of these subsets to a vertex in the 
other subset. For example, consider the graph representing marriages between people in 
a village, where each person is represented by a vertex and a marriage is represented by 
an edge. In this graph, each edge connects a vertex in the subset of vertices representing 
males and a vertex in the subset of vertices representing females. This leads us to the 
following definition. 


DEFINITION 5. A s imp le graph G is c all ed bipartite if its vertex set V can be 
partitioned into two disjoint nonempty sets Ft and Vi such that every edge in the 
graph connects a vertex in Fg and a vertex in Vi (so that no edge in G connects 
either two vertices in Fg or two vertices in Vi). 


In Example 8- we will show that Q, is bipartite, and in Example 9 we will show 
that K>, is not bipartite. 


Ce is bipartite, as shown in Figure 7. since its vertex set can be partitioned into the two 
sets Fg = {vi, vy, V 5 } and V7 - {vj, v 4 , Vf,}. and every edge of Q, connects a vertex in 
Fg and a vertex in Fj. ■ 


Kj is not bipartite. To see this, note that if we divide the vertex set of K$ into two disjoint 
sets, one of the two sets must contain two vertices. If the graph were bipartite, these two 
vertices could not be connected by an edge, but in K\ each vertex is connected to every 
other vertex by an edge. ■ 



FIGURE 7 Showing That C* Is Bipartite. 
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EXAMPLE 10 


EXAMPLE 11 


EXAMPLE 12 



b 



d 


G a 

FIGURE 8 The Undirected Graphs G and H. 

Are [he graphs G and H displayed in Figure 8 bipartite? 

Solution: Graph G is bipartite, since its vertex: set is the union of two disjoint sets, 
{a, b, d} and {c, e, f , g}. and each edge connects a vertex in one of these subsets to a 
vertex in the other subset, (Note that for G to be bipartite it is not necessary that every 
vertex in {a, b d) be adjacent to every vertex in {c, e, /, g}. For instance, b and g are not 
adjacent.) 

Graph H is not bipartite since its vertex set cannot be partitioned into two subsets so 
that edges do not connect two vertices from the same subset. (The reader should verify 
this by considering the vertices a, b, and/.) ■ 


Complete Bipartite Graphs The complete bipartite graph K„ in is the graph that has 
its vertex set partitioned into two subsets of m and n vertices, respectively. There is an 
edge between two vertices if and only if one vertex is in the first subset and the other 
vertex is in the second subset. The complete bipartite graphs XV;, K\ j, K u . and Kif, 
are displayed in Figure 9. I 

SOME APPLICATIONS OF SPECIAL TYPES OF GRAPHS 


We will show how special types of graphs are used in models for data communications 
and parallel processing. 


Local Area Networks The various computers in a bui Iding, such as mi nicomputers and 
personal computers, as well as peripheral devices such as printers and plotters, can be 




t IGURE ? Some Complete Bipartite Graphs. 
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EXAMPLE 13 


connected using a local, area network. Some of these networks are based ona.rfar topology, 
where all devices are connected to a central control device. A local area network can be 
represented using a complete bipartite graph A'i „, as shown in Figure 10(a). Messages are 
sent from device to dev ice through the central control device. 

Other local area networks are based on a ring topology, where each device is con¬ 
nected to exactly two others. Local area networks with a ring topology are modeled 
using B-cycles. C n , as shown in Figure 10(b). Messages are sent from device to device 
around the cycle until the intended recipient of a message is reached. 

Finally, some local area networks use a hyhrid of these two topologies. Messages 
may be sent around the ring, or through a centra) device. This redundancy makes the 
network more reliable. Local area networks with this redundancy can be modeled using 
wheels Wr. as shown in Figure 10(c). El 


Interconnection Networks for Parallel Computation Until recently, computers exe¬ 
cuted programs one operation at a time. Consequently, the algorithms written to solve 
problems were designed to perform one step at a time; such algorithms arc called 
serial. (Almost all algorithms described in this book are serial.) However, many com¬ 
putationally intense problems, such as weather simulations, medical imaging, and 
cryptanalysis, cannot be solved in a reasonable amount of time using serial operations, 
even on a supercomputer. Furthermore, there is a physical limit to how fast a computer 
can carry out basic operations, so that there will always be problems that cannot be 
solved in a reasonable length of time using serial operations. 

Parallel processing, which uses computers tnade up of many separate processors, 
each with its ow n memory', helps overcome the limitations of serial computers. Parallel 
algorithms, which break a problem into a number of subproblems that can be solved 
concurrently, can then be devised to rapidly solve problems using a computer with 
multiple processors. In a parallel algorithm, a single instruction stream controls the 
execution of the algorithm, sending subproblems to different processors, and directs 
the input and output of these subproblems to the appropriate processors. 

When parallel processing is used, one processor may need output generated by an¬ 
other processor. Consequently, these processors need to be interconnected. We can use 
the appropriate type of graph to represent the interconnection network of the processors 
in a computer with multiple processors. In the following discussion, we will describe 
the most commonly used types of interconnection networks for parallel processors. The 
type of interconnection network used to implement a particular parallel algorithm de¬ 
pends on the requirements for exchange of data between processors, the desired speed, 
and, of course, the available hardw are. 

The simplest, but most expensive, network-interconnecting processors include a 
two-way link between each pair of processors. This network can be represented by 



Ml lb) (cl 

FIGURE 10 Star, Ring, and Hybrid Topologies for l ocal Area Networks. 
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r, A A f g 

FIGURE 11 A Linear Array for Six Processors. 

A'„. the complete graph on n vertices, when there are n processors. However, there 
are serious problems with this type of interconnection network because the required 
number of connections is so large. In reality, the number of direct connections to a 
processor is limited, so when there are a large number of processors, a processor cannot 
be linked directly to all others. For example, when there are 64 processors, C(64,2) = 
2016 connections would be required, and each processor would have to be directly 
connected to 63 others. 

On the other hand, perhaps the simplest way to interconnect n processors is to 
use an arrangement known as a linear array. Each processor P,, other than Fj and 
P n , is connected to its neighbors P s -] and P i+ \ via a two-way link. Pi is connected 
only to Pi, and P n is connected only to P„-\. The linear array for six processors is 
shown in Figure 11, The advantage of a linear array is that each processor has at most 
two direct connections to other processors. The disadvantage is that it is sometimes 
necessary to use a large number of intermediate links, called hops, for processors to 
share information. 

The mesh network (or two-dimensional array) is a commonly used intercon¬ 
nection network. In such a network, the number of processors is a perfect square, say 
n = m l . The n processors are labeled P(i, j), 0 < i < m - 1, 0 < j < m - 1. Two- 
way links connect processor P(i, j ) with its four neighbors, processors P(i + 1, j) and 
P(i, j 1 1), as long as these are processors in the mesh. (Note that four processors, on 
the comers of the mesh, have only two adjacent processors, and other processors on the 
boundaries have only three neighbors. Sometimes a variant of a mesh network in which 
every processor has exactly four connections is used; see Exercise 44 at the end of this 
section.) The mesh network limits the number of links_for each processor. Communi¬ 
cation between some pairs of processors requires 0(Jn) = Q(rn) intermediate links. 
(See Exercise 45 at the end of this section,) The graph representing the mesh network 
for 1.6 processors is shown in Figure 12. 

One important type of interconnection network is the hypercube. For such a net¬ 
work, the number of processors is a power of 2, n = 2 m . The n processors are la¬ 
beled Pq, Pi . P„- 1 . Each processor has two-way connections to m other processors. 

Processor P._ is linked to the processors with indices whose binary representations dif¬ 
fer from the binary representation of i in exactly one bit. The hypercube network bal¬ 
ances the number of direct connections for each processor and the number of inter¬ 
mediate connections required so that processors can communicate. Many computers 
have been built using a hypercube network, and many parallel algorithms have been 


A0,i); Pit), 1) AO. 2i m 3) 

K2,?l 

0 ( 3 , 3 ) 

FIGURE 12 A Mesh Network for 16 Processors. 
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EXAMPLE 14 



FIGURE 13 A Hvpereube Network for Eight Processors. 


devised that use a hypercube network. The graph Q„, the n-eube, represents the 
hypercube network with n processors. Figure 13 displays the hypercube network for 
eight processors. (Figure 13 displays a different way to draw {>3 than was shown in 
Figure 6 .) 

■ 


NEW GRAPHS FROM OLD 

Sometimes we need only part of a graph to solve a problem. For instance, we may care 
only about the part of a large computer network that involves the computer centers in 
New York, Denver, Detroit, and Atlanta. Then we can ignore the other computer centers 
and all telephone lines not linking two of these specific four computer centers. In the 
graph model for the large network, we can remove the vertices corresponding to the 
computer centers other than the four of interest, and we can remove all edges incident 
with a vertex that was removed. When edges and vertices are removed from a graph, 
without removing endpoints of any remaining edges, a smaller graph is obtained. Such 
a graph is called a subgraph of the original graph. 


DEFINITION 4 A subgraph of a graph G = (V,E)is. agrapfaif ~ (W, 
WQVasdFQE. 


The graph G shown in Figure 14 is a subgraph of K$. 


Two or more graphs can be combined in various ways. The new graph that contains 
all the vertices and edges of these graphs is called the union of the graphs. We will give 
a more formal definition for the union of two simple graphs. 


DEFINITION 7. The union of two sintple graphs G\ * (V'j. Ej) and G 2 = 
(V 2 , Ei) is the simple graph with vertex set Vf UV3 tad edge aet-Si US*. The union 
of Gi amt Gi is denoted by Gj U Gi. 



FIGURE 14 A Subgraph of K$. 


b 
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EXAMPLE 15 Find the union of the graphs Gj and Gj shown in Figure 15{a). 

Solution: The vertex set of the union G\ UG’j is the union of the two vertex sets, namely, 
{<:j, b. c, d. e, /}, The edge set of the union is the union of the two edge sets. The union 
is displayed in Figure 15(b). ■ 


Exercises 


In Exercises 1 ,5 tint! the number of vertices, the number of 
edges, and die degree of each vertex in the given undirected 
graph, Identify all isolated and pendant vertices, 

1, « b r 



4. Find the sum of the degrees of the vertices of each 
graph in Exercises 1-3 and verify that it equals twice 
the number of edges in the graph. 

5. Can a simple graph exist with 15 vertices each of de¬ 
gree 5 11 

6. Show that the sum, over the set of people at a party, of 
the number of people a person has shaken hands with, 
is even. Assume that no one shakes his or her own hand. 

In Exercises 7-9 determine the number of vertices and 
edges and tint! the in-degree and out-degree of each vertex 
for the given directed multigraph. 
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10, For each of the graphs in Exercises 7-9 determine the 
sum of the in-degrees of the vertices and the sum of the 
out-degrees of the vertices directly. Show that they arc 
both equal to the number of edges in the graph. 

11, Construct the underlying undirected graph for the 
graph with directed edges in Figure 2. 

12, Draw the following graphs. 

a) K, hi A , vi A,, 

d) ('■ e) H fi & 


In Exercises 13-17 determine whether the graph is bi¬ 
partite. 



21. Does there exist a simple graph with five vertices of the 

following degrees : 1 If so. draw such a graph. 

a) 3, 3, 3, 3,2 b) 1, 2,3, 4.5 

c) 1. 2 , 3, 4, 4 d) 3, 4, 3, 4. 3 

e) 0, 1,2, 2,3 f) 1. 1, 1. 1, 1 

22. Flow many subgraphs with at least one vertex does A3 
have? 

23. How many subgraphs with at least one vertex docs A? 
have? 

24. How many subgraphs with at least one vertex does Ms. 
have? 

25. Draw all subgraphs of the following graph. 


a b 



d 


26. Let (7 be a graph with v vertices and e edges. Let M be 
the maximum degree of the vertices of G, and let m be 
the minimum degree of the vertices of G. Show that 
a) 2e/v > m. b) 2e/v £ M. 


(6. a 



A simple graph is called regular if every vertex of this 
graph has the same degree. A regular graph is called n- 
regular if every' vertex in this graph has degree n. 

27. For which values of n are the following graphs regular? 
a) K„ b) C„ c) W„ d) 

28. For which values of m and n is K n „ regular? 

29. How many vertices does a regular graph of degree 4 
with 10 edges have? 


17. 


18. For which values of n are the following graphs bi¬ 
partite? 

a) K, b) C’„ e) W„ d) Q„ 

19. How many vertices and how many edges do die follow ¬ 
ing graphs have? 

a) A, h) C„ c) W„ 

d) A„,„ e) Q„ 

2(1. How many edges does a graph have if it has vertices of 
degree 4. 3, 3. 2, 2? Draw' such a graph. 


In Exercises 30-32 find the union of the given pair of sim¬ 
ple graphs. (Assume edges with the same endpoints are the 
same.) 

30. a 



d . d 


31. a b a f h 



r 4 £ g d 
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The converse of a directed graph G = {V’ E'y denoted by 
G c , is the directed graph (V, F ) where («, v) £ F if and only 
if (y, u) £ E. 

39. Draw the converse of each of the graphs in Exercises 
7-9 in Section 7.1. 

40. Show that {G‘ : ) c = G whenever G is a directed graph. 

41. Show that the graph G is its own converse if and only 
if the relation associated with G (see Section 6.3) is 
symmetric, 

42. Extend (he definition of the converse of a directed 
graph to the notion of the converse of tt directed multi- 
graph. 

43. Draw the mesh network for interconnecting nine par¬ 
allel processors. 

44. In a variant of a mesh network, for interconnecting 
n = m 1 processors, processor P(i, j) is connected to 
the four processors PIU ± 1) mod m, j ), P(i, (j ± 1) 
mod m), so that connections wrap around the edges of 
the mesh. Draw this variant of the mesh network for 16 
processors. 

45. Show that every 1 pair of processors in a mesh netw ork of 
n ~ m‘ processors can communicate using 0( v rt) = 
0{ m ) hops between directly connected processors. 

73 _____________ 

Representing Graphs and Graph Isomorphism 

INTRODUCTION 


There are many useful ways to represent graphs. As we w ill see throughout this chapter, 
in working with a graph it is helpful to be able to choose its most convenient represen¬ 
tation. In this section we wall show how' to represent graphs in several different ways. 
Sometimes, two graphs have exactly the same form, in the sense that there is a 
one-to-one correspondence between their vertex sets that preserves edges. In such a 
case, we say that the two graphs are isomorphic. Determining whether two graphs 
are isomorphic is an important problem of graph theory that we will study in this section. 

REPRESENTING GRAPHS 


One way to represent a graph without multiple edges is to list all the edges of this 
graph. .Another way to represent a graph with no multiple edges is to use adjacency 
lists, which specify the vertices that are adjacent to each vertex of the graph, 

EXAMPLE I Use adjacency lists to describe the simple graph given in Figure 1. 

Solution: Table 1 lists those vertices adjacent to each of the vertices of the graph. ■ 

EXAMPLE 2 Represent the directed graph shown in Figure 2 by listing all the vertices that are the 

terminal vertices of edges starting at each vertex of the graph. 



33. The complementary graph G of a Simple graph G has 
the same vertices as G. Two vertices are adjacent in 
G if and only if they are not adjacent in G. Find the 
following. 

at T n b) 

o c~ d) a 

34. If G is a simple graph with 15 edges and G has 13 
edges, how many vertices does G have? 

35. If the simple graph G has v vertices and e edges, how 
many edges does 6’ have? 

*36. Show that if G is a bipartite simple graph with v ver¬ 
tices and e edges, then e s v 2 /4 
37. Show that if G is a simple graph with n vertices, then 
the union of G and G is A'„. 

*38. Describe an algorithm to decide whether a graph is 
bipartite 
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b 



FIGURE 1 A Simple Graph. 



TABI.F. 1 An Edge List for a 
Simple Graph. 

Vertex 

Adjacent Vertices 

Q 

t\ c, e 

b 

a 

<■ 

«. fd e 

d 

r. e 

e 

a, c, d 


TABLE 2 An Edge List for a 

Directed Graph. 

Initial Vertex 

Terminal Vertices 

a 

b, c, d. e 

b 

b> d 

c 

a, c, £? 

d 


e 

b, c, d 


Solution: Table 2 represents the directed graph shown in Figure 2. ■ 


ADJACENCY MATRICES 


Carrying out graph algorithms using the representation of graphs by lists of edges, or by 
adjacency lists, can be cumbersome if there are many edges in the graph. To simplify 
computation, graphs ears be represented using matrices. Two types of matrices com¬ 
monly used to represent graphs will be presented here. One is based on the adjacency 
of vertices, and the other is based on incidence of vertices and edges. 

Suppose that G = ( V, E) is a simple graph where | F| = n. Suppose that the vertices 

of C are listed arbitrarily as i*i, vj.v„. The adjacency matrix A (or Ag) of G, with 

respect to this listing of the vertices, is the n x n zero-one matrix with l as its (ii yjth 
entry when v, and v, are adjacent, and 0 as its (i, yjth entry' when they are not adjacent. 
In other words, if its adjacency matrix is A = IX,], then 


1 if {vj, vj } is an edge of G\ 
0 otherwise. 


Note that an adj acency matrix of a graph is based on the ordering chosen for the vertices. 
Hence, there are as many as«' different adjacency matrices for a graph with r vertices, 
since there are «! different orderings of n vertices. 

The adjacency matrix of a simple graph is symmetric, that is, a;. = a ;i , since both 
of these entries are 1 when v< and v, are adjacent, and both are 0 otherwise. Furthermore, 
since a simple graph has no loops, each entry an, i = 1 , 2,3, .... n, is 0. 

Note that when there are relatively few edges in a graph, the adjacency matrix is a 
sparse matrix, that is, a matrix with few nonzero entries. There are special techniques 
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Car a'pivxx-jitiivg. ami cprvipuntig with. such matrices. Also, it sometimes may be more 
silicic.]: m work with lists of edges when representing and working with such graphs. 

i:\A\u-u::< 

Use an adjacency matrix to rcpivsenr the graph shown in Figure 3. 

— * 

, \ 

Sothium. We order the vertices as j. />. r. d. The matrix representing this graph is 

R> 1 1 11 

i 0 1 (i 

■i 

1 1 0 SI | 

i (i on 1 - m 

1 (C! Kl J A 
Simple Graph. 

LXAMPi.h-t 

Draw a graph with the adjacent;)- matrix 

* 

ir - 

['0 1 l 0" 

1 " » '! 

! 0 0 1 

|0 i 1 0, 

with respect to the ordering of ceil tees a. b, t, if. 

StMiMmi: A graph with this adjacency matrix i.s .shown in Figure 4. ■ 

F1GHKK4 A 

Graph with fltt L 
Given Adjacent) 
Matrix. 

Adjacency matrices can also he used to represent undirected graphs with loops and 
with multiple edges. A loop at the vertex a, is represented by a 1 at the (/, /)tb position 
ai the adjacency matrix. When multiple edges are present, the adjacency matrix is no 
longer a ?cru one matrix, since the ft, j)th entry of this matrix equals the number of 
edges that are associated to {«,, tijJ. All undirected graphs, including multigraphs and 
pseudographs. have symmetric adjacency matrices. 

KXAMPI.K? 

the an adjacency matrix to represent the pseudograph shown in Figure 5. 

j j 

.* 

Sohtuva. The adjacency matrix using the ordering of vertices a, b, r, d is 

ro 3 o ii 

! 3 0 1 l 

In i i 2 1' 


'dll O' ■ 

i-tgir!-:? t 
Pxeudugrapli. 

We used zero-one matrices in Chapter 6 to represent directed graphs. The matrix 
lor a directed graph 0 - (V. E) lias a I m its ui /)th position if there is an edge from v, 

to i' ; . where r ( , vy.i.s an arbitrary listing of the vertices of the directed graph. In 

other words, it A -- Ja (/ j is the adjacency matrix for the directed graph with respect to 
this listing of the vertices, then 

.. i I if ftp v,) is an edge of G. 

\ 0 otherwise. 
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EXAMPLE 6 


EXAMPLE 7 


The adjacency matrix for a directed graph does not have to be symmetric, since there 
may not be an edge from a , to a. when there is an edge from at to a } . 

Adjacency matrices can also be used to represent directed multigraphs. Again, such 
matrices are not zero-one matrices when there are multiple edges in the same direction 
connecting two vertices. In the adjacency matrix for a directed multigraph, a :f equals 
the number of edges that are associated to (>y, vy). 


INCIDENCE MATRICES 


Another common way to represent graphs is to use incidence matrices. LetG = (V, E) 

be an undirected graph. Suppose that v \, vj,..,, v n are the vertices and e \, . t' m are 

the edges of G. Then the incidence matrix with respect to this ordering of V and E is 
the n X m matrix M = [m, /]. where 

_ j 1 when edge ey is incident with v,-, 
u 10 otherwise. 


Represent the graph shown in Figure 6 with an incidence matrix. 

Solution: The incidence matrix is 
ty fy ey e<j 

v, f 1 I 0 0 0 O' 

v-2 0 0 1 1 0 1 

v 3 0 0 0 0 1 1 . 

v 4 1 0 I 0 0 0 


Incidence matrices can also be used to represent multiple edges and loops, Multiple 
edges are represented in the incidence matrix using columns with identical entries, since 
these edges are incident with the same pair of vertices. Loops are represented using a 
column with exactly one entry equal to 1, corresponding to the vertex dial is incident 
with this loop. 


Represent the pseudograph shown in Figure 7 using an incidence matrix, 


v 2 f 6 



FIGURE 6 An Undirected Graph. 
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Solution: The incidence matrix for this graph is 


e\ e 2 

v, f 1 ] 

v 2 0 1 

v 3 0 0 

v 4 0 0 

vs 0 0 


< i *4 <? 3 

1 0 0 

1 1 0 

0 1 1 

0 0 0 

0 0 1 


e? I'a 
0 0 0 ' 

1 L 0 

0 0 0 

0 I 1 

1 0 0 



ISOMORPHISM OF GRAPHS 


We often need to know whether it is possible to draw two graphs in the same way. For 
instance, in chemistry, graphs are used to model compounds. Different compounds can 
have the same molecular formula but can differ in structure. Such compounds will be 
represented by graphs that cannot be drawn in the same way. The graphs represent¬ 
ing previously known compounds can be used to determine whether a supposedly new 
compound has been studied before. 

There is a useful terminology for graphs with the same structure. 

DEFINITION 1. The simple graphs Gj = (V l , £j) and G% = (V^ £ 2 ) a*® iso¬ 
morphic if there is a one-to-one and onto function/ from V\ to Vj with the property 
that a and b are adjacent in Gi if and only if /(a) and fib) are adjacent in Gj, for 
all a and b in Pi . Such a function / is called an isomorphism.* 


H 

FIGURES The In other words, when two simple graphs are isomorphic, there is a one-to-one cones pon- 
Graphs G and H. dence between vertices of the two graphs that preserves the adjacency relationship. 


EXAMPLES Show that the graphs (i = (V, E) and H = (IE F), displayed in Figure 8 , are iso¬ 

morphic. 

Solution: The function / with f{u{) - vi, f(uf) - v 4 , /(«j) = rj, and /\« 4 ) - tw 
is a one-to-one correspondence between V and IE. To see that this correspondence pre¬ 
serves adjacency, note that adjacent vertices in G are u } and 1 * 2 , u ; and mj. hi and a 4 . 
and u-< and « 4 . and each of the pairs /(>;) = vj and /(« 2 ) = v 4 , /(«j) = v ( and 
fiuf) = v 3> f(ui) - v 4 and /f« 4 ) = i j i, and f(u{) = vjand fiu 4 ) - are adjacent 
in H. m 


It is often difficult to determine whether two simple graphs are isomorphic. There 
web are rtf possible one-to-one correspondences between the vertex sets of two simple 
graphs with n vertices. Testing each such correspondence to see whether it preserves 
adjacency and ntmadjacency is impractical if n is at all large. 

However, we can often show that two simple graphs are not isomorphic by showing 
that they do not share a property that isomorphic simple graphs must both have. Such 
a property is called an invariant with respect to isomorphism of simple graphs. For 


’The word isomorphism comes from the Greek roots iaa for ''equal” and morphe fur "form, ' 
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H 


FIG URE 9 The Graphs G and H. 


instance, isomorphic simple graphs must have the same number of vertices, since there 
is a one-to-one correspondence between the sets of vertices of the graphs. Furthermore, 
isomorphic simple graphs must have the same number of edges, because the one-to- 
one correspondence between vertices establishes a one-to-one correspondence between 
edges. In addition, the degrees of the vertices in isomorphic simple graphs must be 
the same, That is, a vertex v of degree d in G must correspond to a vertex /(v) of 
degree d in H. since a vertex w in G is adjacent to v if and only if f(v) and f(w) are 
adjacent in H. 


LXAMPLK 9 


Show that the graphs displayed in Figure 9 are not isomorphic. 


Solution: Both G and H have five vertices and six edges. However, H has a vertex of 
degree 1, namely, e, w'hereas G has no vertices of degree 1. It follows that G and H are 


not isomorphic. 


The number of vertices, the number of edges, and the degrees of the vertices are 
all invariants under isomorphism. If any of these quantities differ in two simple graphs, 
these graphs cannot be isomorphic. However, when these invariants are the same, it 
does not necessarily mean that the two graphs are isomorphic. There are no useful sets 
of invariants currently known that can be used to determine whether simple graphs are 
isomorphic. 


LXAMPI.L Hi Determine whether the graphs shown in Figure 10 are isomorphic. 


h 



H 


FKiURL HI The Graphs G and H. 
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/ h 


/ h 


\ 


* 


FIGURE II The 
Subgraphs ttf ti 
and H Made Up 
of Vertices of 
Degree 3 and the 
Edges Connecting 
Them, 


EXAMPLE II 


Solution: The graphs G and H both have eight vertices and JO edges. They also hath 
have four vertices of degree 2 and four of degree 3, Since these invariants all agree, it 
is still conceivable that these graphs are isomorphic. 

However, G and H are not isomorphic. To see this, note that since degta) - 2 in 
G. a must correspond to either i, u, a; or y in II, since these are the vertices of degree 2 
in H. However, each of these four vertices in H is adjacent to another vertex of degree 
2 in H, which is not true for a in G. 

Another way to see that G and H are not isomorphic is to note that the subgraphs 
of G and H made up of vertices of degree 3 and the edges connecting them must be 
isomorphic if these two graphs are isomorphic (the reader should verify this), However, 
these subgraphs, shown in Figure I i, are not isomorphic. ■ 


To shew that a function / from the vertex set of a graph G to the vertex set of a 
graph H is an isomorphism, we need to show that/ preserves edges. One helpful way 
to do this is to use adjacency matrices. In particular, to show that/ is an isomorphism, 
we can show that the adjacency matrix of G is the same as the adjacency matrix of H, 
when raws and columns are labeled to correspond to the images under/ of the vertices 
in G that are the labels of these rows and columns in the adjacency matrix of G. We 
illustrate how this is done in the following example. 


Determine whether the graphs 6’ and H displayed in Figure 12 are isomorphic. 

Stibium: Both C and H have six vertices and seven edges, Both have four vertices of 
degree 2 and two vertices of degree 3. It is also easy to see that the subgraphs of G and 
H consisting of all vertices of degree 2 and the edges connecting them are isomorphic 
(as the reader should verify). Since G and H agree with respect to these invariants, it is 
reasonable to try to hnd an isomorphism/. 

We now will define a function/ and then determine whether it is an isomorphism. 
Since deg(«i) = 2 and since ip is not adjacent to any other vertex of degree 2, the 
image of m must be either V 4 or vy,, the only vertices of degree 2 in H not adjacent 
to a vertex of degree 2, We arbitrarily set f(u 1 ) = vy,. [If we found that this choice 
did not lead lu isomorphism, we would then try /(hi) - v 4 J Since tt-> is adjacent to 
M], the possible images of u-> are and iy. We arbitrarily set /(«;') — rv Coniimiing 
in this way, using adjacency of vertices and degrees as a guide, we set - r 4 , 
fin 4 ) - is, J (ms) = V|, and /(«;,) = vy We now have a one-to-one correspondence 
between the vertex set of G and the vertex set of H, namely: /(jp ) - %, fiuG ~ r ?- 
fin. A = V 4 , /(M 4 I = V 5 , fiMs) — v;, /(Ms) = vi. To sec whether/ preserves edges. 



_G_ H 

FIGURE 12 GraphsG and ff. 
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we examine the adjacency matrix of G, 

U\ Hi U} Hj Mi lit, 

Hi [0 1 0 1 0 o' 

Hi 1 0 1 0 0 1 

. = Hi 0 10 10 0 

‘ ° «4 1 0 l 0 1 0 ' 

Hi 0 0 0 1 0 1 

Hh i 0 1 0 0 1 0 j 

and the adjacency matrix of H with the rows and columns labeled by the images of the 
corresponding vertices in G . 



Vf, 

Vl 

V4 

v? 

Vi 

V> 

V(i 

"0 

1 

0 

i 

0 

01 

V} 

1 

0 

1 

0 

0 

1 ! 

i - l 4 

A u - 

n 

0 

1 

1 

0 

0 

1 

1 

0 

0 

1 

0 

0 

Vi 

0 

0 

0 

1 

0 

1 

V2 

0 

1 

0 

0 

1 

0 ! 


Since A (l - - A H . it follows that/ preserves edges. We conclude that / is an isomor¬ 
phism, so that G and H are isomorphic. Note that if f turned out not to be an isomor¬ 
phism, we would not have established that G and H are not isomorphic, since another 
correspondence of the vertices in G and H may be an isomorphism. ■ 


The best algorithms known for determining whether two graphs are isomorphic 
mb have exponential worst-case time complexity (in (he number of vertices of the graphs). 
However, linear average-case time complexity algorithms are known lhat solve this 
problem, and there is some hope that an algorithm with polynomial worst-case time 
complexity for determining whether two graphs are isomorphic can be found. The 
best practical algorithm, called NAUTY, can be used to determine whether two graphs 
with as many as 100 vertices are isomorphic in less than 1 second on a modem PC. 
The software for NAUTY can be downloaded over the Internet and experimented 
with. 


Exercises 
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6. Represent the graph in Exercise 2 with an adjacency 
matrix.. 

7. Represent the graph in Exercise 3 with an adjacency 
matrix. 

8. Represent the graph in Exercise 4 with an adjacency- 
matrix. 

4. Represent each of the following graphs with an adja¬ 
cency matrix, 
a) A'., ft i A 4 
c) Ay _) dt r , 

el W 4 fi O 

In Exercises 10--12 draw a graph with the given adjacency 
matrix. 

10. j'D ! Oj 
11 0 I i 

; 0 i 0_ 


0 

0 1 

ll 

12. 

') 1 

o' 

0 

0 1 

0 


f) 0 

[) 

1 

1 0 

lj 


1 0 

0 

1 

1 1 

0. 


1 1 

0 ; 


In Exercises 1 3 15 represent the given graph using an 
adjacency matrix. 

13. 0 h 

T A 

/ : \ 



a 


14. ii -_-_ft 




15. 




i i X ; i 
I !/ \\ / 

A M 


In Exercises 16-18 draw an undirected graph represented 
by the given adjacency matrix. 


16. fl 

l , 


0 4 
4 0 


17* /"] 2 0 1 j 

1 2 G 3 01 
!o 3 1 lj 

[l 0 1 Oj 


18. 10 1 3 0 4“ 

I 2 1 3 0 : 

3 1 1 0 1 | 

|0 3 0 0 2 j 

■4 0 1 2 3. 


In Exercises 10-21 find the adjacency matrix of the given 
directed multigraph. 



In Exercises 22-24 draw the graph represented by the given 
adjacency matrix. 


22. M 0 r 

jo 0 1 


\2 0 Oj 

■0 2 2j 


-4. 2 3 0] 

1 3. 2 1 I 

2 1 1 0 
1 0 0 2 J 

25. Is every zero-one square matrix that is symmetric and 
has zeros on the diagonal the adjacency matrix of a si Ri¬ 
fle graph? 

26. Use an incidence matrix to represent the graphs iti 
Exercises ) and 2. 

27. Use an incidence matrix to represent the graphs in 
Exercises 13-15. 


*28. What is the sum of the entries in a row of the adjacency 
matrix for an undirected graph? Fora directed graph'* 
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*29. What is the sum of the entries in a column of the 
adjacency matrix for an undirected graph? For a di¬ 
rected graph? 

30. What is the sum of the entries in a row of the incidence 
matrix for an undirected graph? 

31. What is the sum of the entries in a column of the inci¬ 
dence matrix for an undirected graph? 

*32. Find an adjacency matrix for each of the following, 
a) K„_ b) C„ c) W„ d) K„,, e) Q„ 
*33. Find incidence matrices for the graphs in parts (aH'd'i 
of Exercise 32, 


In Exercises 34—44 determine whether the given pair of 
graphs is isomorphic, 
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44. 




45. Show that isomorphism of simple graphs is an equiva¬ 
lence relation. 

46. Suppose that G and H are isomorphic simple graphs. 
Show that their complementary graphs G and ft are 
also isomorphic. 

47. Describe the row and column of an adjacency matrix 
of a graph corresponding to an isolated vertex. 

48. Describe the row of at) incidence matrix of a graph cor¬ 
responding to an isolated vertex. 

49. Show that the vertices of a bipartite graph with two or 
more vertices can be ordered so that its adjacency ma¬ 
trix lias the form 

1° A l 

[B OJ 

where the four entries shown arc rectangular blocks. 

A simple graph G is called self-complementary if G and 
(} are isomorphic. 

5(1. Show that the following graph is self-complementary 
a b 


i i 

ll C 


a) 

0 

0 

11 ro 

i 

1 



io 

0 

'M 1 

0 

0 



[i 

1 

OJ [i 

0 

0. 



bl 

0 

1 

0 1 

io 

1 

1 

l' 


1 

0 

0 1 


i 

0 

0 

1 


0 

0 

0 1 

p 

i 

0 

0 

1 


1 

1 

1 0 


i 

1 

1 

0 

c) 

0 

1 

1 0 


0 

1 

0 

1 i 


1 

0 

0 1 


1 

0 

0 

0 : 


1 

0 

0 1 


0 

0 

0 

1J 


0 

1 

1 0 


1 

0 

1 

°J 


58. Determine whether the graphs without loops with the 
following incidence matrices are isomorphic. 


a) 

'1 0 11 

'1 1 0’ 


o i iL 

1 0 1 


.1 1 OJ 

0 1 lj 


b) 

1 

1 

0 

0 

0 


0 

1 

0 

0 



1 

0 

1 

0 

1 


0 

J 

1 

1 

0 


0 

0 

0 

1 

1 


1 

0 

0 

1 

0 


0 

1 

1 

1 

0 


1 

0 

1 

0 

1 


59, Extend the definition of isomorphism of simple graphs 
to undirected graphs containing loops and multiple 
edges. 

60. Define isomorphism of directed graphs. 

In Exercises 61-64 detenuine whether the given pair of 
directed graphs is isomorphic. 



51. Find a self-complementary simple graph with five, 
vertices. 

*52. Show that if G is a self-complementary simple graph 
with v vertices, then v = 0 or I (mod 4). 

53. For which integers n is C„ self-complementary? 

54. How many nonisomorphic simple graphs are there with 
n vertices, when n is 

a) 2 > b) 37 e) 47 

55. How many nonisomorphic simple graphs are there with 
five vertices and three edges? 

56. How many nonisomorphic simple graphs are there with 
.six vertices and four edges? 

57. Are ihe simple graphs with the following adjacency 
matrices isomorphic? 



63. 



a i 
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64. u | 



65. Show that if (i and fi are isomorphic directed graphs, 
(hen the converses of 0 and fi (defined in the preamble 
of Exercise 39 of Section 7.2) are also isomorphic. 


*66, How many non isomorphic directed simple graphs are 
there with n vertices, when n is 

a) 2 ! 

b) 3? 

C) 4? 

*67. What is the product of the incidence matrix and its. 

transpose for an undirected graph? 

*68. How much storage is needed to represent a simple 
graph with v vertices and e edges using 
a) adjacency lists? 
h) an adjacency matrix? 

c) an incidence matrix? 

A devil’s pair for a purported isomorphism test is a pair 
of nonisomorphic graphs that the lest fails to show arc not 
isomorphic. 

69. Find a devil's pair for the test that checks the sequence 
of degrees of the vertices in the two graphs to make 
sure they agree, 


7 . 4 _ 

Connectivity 


INTRODUCTION 

Many problems can be modeled with paths formed by traveling along the edges of 
graphs. For instance, the problem of determining whether a message can be sent be¬ 
tween two computers using intermediate links can be studied with a graph model. Prob¬ 
lems of efficiently planning routes for mail delivery, garbage pickup, diagnostics in 
computer networks, and so on, can be solved using models that involve paths in graphs 


PATHS 


We begin by defining the basic terminology uf graph theory that deals with paths, 


DEFINITION 1. A path of length n from u to v, where n is a positive integer, 

in an undirected graph is a sequence of edges e\ _ ,e n of the graph such that 

f(e t) = {*a, Jfi>, f(ei) = {xi, xi},...,/(e„) - *„}, where x 0 = u and 

x H = v. When the graph is simple, we denote this path by its vertex sequence 

.% xi_x n (since listing these vertices uniquely determines the path). The path is 

a circuit if it begins and ends at the same vertex, that is, if u = v. The path or circuit 
is said to pass through or traverse the vertices X\, x 2 , ,.., A path or circuit is 
simple if it does not contain the same edge more than once. 


When it is not necessary to distinguish between multiple edges, we will denote a 

path C], ci . e„ where f(ej) - {x ( _i, x- t \ for i - 1,2,..., n by its vertex sequence 

xt\, X|.a„. This notation identifies n path only up to the vertices it passes through. 

There may be more than one path that passes through this sequence of vertices 
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EXAMPLE 1 


a b c 



d e f 


FIGURE 1 A Simple Graph. 


In the simple graph shown in Figure 1, a, d. e is a simple path of length 4, since 
[a, d\, {J, r}, {c, /}, and {/, e] are all edges. However, d, e, c, a is not a path, since 
{e, c} is no! an edge. Note that h, c, /, e, b is a circuit of length 4 since \b, <:}, {c, /}, 
{/, e}, and {e, b} are edges, and this path begins and ends at b. The path a, b, e, d. a, b. 
which is of length 5, is not simple since it contains the edge {a, b} twice. ■ 


Paths and circuits in directed graphs were introduced in Chapter 6, We now define 
such paths for directed multigraphs. 

DEFINITION 2. A path of length n, where n is a positive integer, from* to 
vist directed moltigraph is a sequence of edges e\, e%... ,e„ of the graph «ch 
that f(e 0 = (xq, xj), f(e 2 ) = (x h xff ../{<?„) « <x„~i,x„), where x* «* trand 
x n - v. When there are no multiple edges in the graph, tfefa path is deserted by its 
vertex sequence xq,, x\, xj, ..., x, A path that begins and cads at the same vertex is 
called a cavuit or cycle. A path nr circuit is called stmplejf it does not contain the 
same edge more than once. 


When it is not necessary to distinguish between multiple edges, wc will denote a 

path ei, t’l.e„ where fie,) = U,.. i, x,) for / = 1.2by its vertex sequence 

xq, .tj. — x„. The notation identifies a path only up to the vertices it passes through 
There may be more than one path that passes through this sequence of vertices. 


CONNECTEDNESS IN UNDIRECTED GRAPHS 

When does a computer network have the property that every pair of computers can share 
information, if messages can be sent through one or more intermediate computers? When 
a graph is used to represent this computer network, where vertices represent the computers 
find edges represent the communications links, this question becomes: When is there 
always a path between two vertices in the graph? 

DEFINITION 3. An undirected graph is called connected if there is a path be¬ 
tween every pair of distinct vertices of the graph. 


Thus, any two computers in the network can communicate if and only if the graph of 
this network is connected. 
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\ / 
\ / 


^- -V 


A /T 


to— 

d 


FIGURE 2 The Graphs G and H. 



FIGURE 3 The Graph G and Its Connected 
ComponentsGi.Gj, and G> 


EX4MP1.E 2 The graph G in Figure 2 is connected, since for every pair of distinct vertices there is 

a path between them (the reader should verify this). However, the graph H in Figure 2 
is not connected. For instance, there is no path in H between vertices <; and d ■ 

We will need the following theorem in Chapter 8. 

THEOREM 1 There is a simple path between every pair of distinct vertices of a connected undi¬ 

rected graph, 

p roo f; Let it and v be two distinct vertices of the connected undirected graph G 

i L. E). Since G is connected, there is at least, one path between u and v. Let , vq , x\ . 

a'„, where jc u = u and x„ = v, be the vertex sequence of a path of least length. This 
path of least length is simple. To see this, suppose it is not simple. Then .v, - xj for 
some i and j with 0 ss i < j. This means that there is a path from u to r of shorter 

length with vertex sequence jcq, xj .,t ( - x , .t„ obtained by deleting the edges 

corresponding to the vertex sequence x„ ,x, j. □ 

A graph that is not connected is the union of two or more connected subgraphs, 
each pair of which has no vertex in common. These disjoint connected subgraphs arc 
called the connected components of the graph. 

EXAMPLE 3 What are the connected components of the graph G shown in Figure 3° 

Solution: The graph G is the union of three disjoint connected subgraphs G\. Gj- 
and G?, shown in Figure 3. These three subgraphs are the connected components 
of G. ■ 


Sometimes the removal of a vertex and all edges incident with it produces a 
subgraph with more connected components than in the original graph. Such vertices 
are called cut vertices (or articulation points). The removal of a cut vertex from 
a connected graph produces a subgraph that is not connected. Analogously, an edge 
whose removal produces a graph with more connected components than in the original 
graph is called a cut edge or bridge. 
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i \ win i 4 


EXAMPLE 5 


hind the cut vertices and cut edges in the graph G shown in Figure 4. 

Solution: The cut vertices of fj arc h , c. anti e. 'I'he removal of one of these vertices (and 
its adjacent edges) disconnects the graph. The cut edges are {a, b] and {c,«}. Removing 
either one of these edges disconnects G. ■ 


CONNECTEDNESS IN DIRECTED GRAPHS 


There are two notions of connectedness in directed graphs, depending on whether the 
directions of the edges are considered. 


DEFINITION 4. A directed graph is strongly connected if there is a pa&irtma 
to b and from b to a whenever a and b are vertices in the graph. 


For a directed graph to be strongly connected there must be a sequence of directed 
edges from any vertex in the graph to any other vertex, A directed graph cart fail to 
he strongly connected hut still he in "one piece." To make this precise, the following 
deli rut ion is given. 

DEFINITION S. A directed graph is wakty cormecte4iS there is a Mthh<fwcen 
any two vertices iu the underlying undirected graph. 


That is, a directed graph is weakly connected if and only if there is always a path 
between two vertices when the directions of the edges arc disregarded. Clearly, anv 
strongly connected directed graph is also weakly connected. 


Are the directed graphs G and H shown in Figure 5 strongly connected? Are they 
weakly connected? 

Solution; G is strongly connected because there is a path between any two vertices in 
this directed graph (the reader should verify this). Hence, G is also weakly connected. 
The graph H is not strongly connected. There is no directed path from a to h in this 
graph, However. H i s weakly connected, since there is a path between any two vertices 
i n the underlying undirected graph of // (the reader should verify this). ■ 


b a h 



fKilkk 4 The Graph G, H(!I;RE5 The Directed Graphs G and//. 
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EXAMPLE 6 


EXAMPLE 7 


PATHS AND ISOMORPHISM 

There are several ways that paths and circuits can help determine whether two graphs 
are isomorphic. For example, the existence of a simple circuit of a particular length 
is a useful in variant that can be used to show that two graphs are not isomorphic:. In 
addition, paths can be used to construct mappings that may be isomorphisms. 

As we mentioned, a useful isomorphic invariant for simple graphs is the existence 
of a simple circuit of length k, where k is a positive integer greater than 2. (The proof 
that this is an invariant is left as Exercise 36 at the end of this section.) Example 6 
illustrates how this invariant can Ire used to show that two graphs me not isomorphic. 


Determine whether the graphs G and H shown in Figure 6 are isomorphic. 

Solution; Both G and H have sis vertices and eight edges. Each has four vertices of 
degree 3. and two vertices of degree 2. So, the three invariants—number of vertices, 
number of edges, and degrees of vertices-—all agree for the two graphs. However. II 
has a simple circuit pf length 3, namely, vj. 14. v- (j . V| whereas G has no simple circuit 
of length 3, as can be determined by inspection (all simple circuits in G have length at 
least 4). Since the existence of a simple circuit of length 3 is an isomorphic invariant, 
G and H are not isomorphic. ■ 


We have shown how the existence of a type of path, namely, a .simple circuit of a 
particular length, can be used to show that two graphs are not isomorphic. We can also 
use paths to find mappings that are potential isomorphisms. 


Determine whether the graphs (} and H shown in Figure 7 are isomorphic, 

Solution: Both G and H have five vertices and six edges, both have two vertices of 
degree 3 and three vertices of degree 2, and both have a simple circuit of length 3, a 
simple circuit of length 4, and a simple circuit of length 5. Since all these isomorphic 
invariants agree. G and H may be isomorphic. To find a possible isomorphism, we can 
follow paths that go through ail vertices so that the corresponding vertices in the two 
graphs have the same degree, For example, the paths ie, «*, uj,, U2< l G in G and m. »o. 


T 


X 


X 




"A 


V| 

A. 

\ 

A-V: 





n ll 

f 1(1( RE ti The GraphsG and//. 
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I HEOKK.V 2 


\ 



J-’IGTRK S The 
Graph C. 


IX ‘Mil I ,S 



G H 

KKil'RE 7 Tlie Graphs G and II. 


i'j, vs, i '4 in H both go through every vertex in the graph, start al a vertex of degree 3, go 
tiiro ugh vertices of degrees 2,3, and 2, respective ly, and end at u vertex of degree 2. By 
following these paths through the graphs, we define the mapping/ with /fui) = vy. 
/'(« 4 ) - i’j. fiuf) — V|, /(>;) — vj, and ftu$) - V 4 . The reader can show that / is an 
isomorphism, so that G and H are isomorphic, either by showing that / preserves edges 
or by showing that with the appropriate orderings of vertices the adjacency matrices of 
G and hi are the same. ■ 


COUNTING PATHS BETWEEN VERTICES 


The number of paths between two vertices in a graph can be determined using Us ad¬ 
jacency matrix 

Let G be a graph with adjacency matrix A with respect to the ordering i'j , vj, ..., v, 
(with directed or undirected edges, with multiple edges and loops allowed). The 
number of different paths of length r from v,- to vj, where r is a positive integer, 
equals the (tj /ith entry of AG 

Proof: The theorem will be proved using mathematical induction. Let G be a graph 

with adjacency matrix A (assuming an ordering r L , v».i „ of the vertices of G). The 

number of paths from vj to v, of length I is the (r, /)th entry' of A. since this entry' is the 
number of edges from v; to v,\ 

Assume that the (/, /)th entry' of A r is the number of different paths of length r from 
Vi to v, This is the induction hypothesis. Since A r+1 = A'A, the (i, /)th entry of A' + ! 
equals 

bmp i + - bpiG, 1 ••• • ■ 

where bn is the (t, A)th entry of AG By the induction hypothesis, bn is the number of 
paths of length r from v,- to ry. 

A path of length r + I from v, to iy is made up of a path of length r from v, to some 
intermediate vertex v*, and an edge from v* to v,-. By the product rule for counting, the 
number of such paths is the product of the number of paths of length r from e, to v t , 
namely, bn, and the number of edges from v* to vj, namely, a ki . When these products 
are added for all possible intermediate vertices v*, the desired result follows by the sum 
rule for counting. □ 


How many [laths of length 4 are (here from a to d in the simple graph G shown in 
figure 8 ? 
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Solution: The adjacency matrix of G (ordering the vertices as a, h, c, d) is 

"01 10 ] 
i 0 0 1 1 
1 0 U : 

oiio! 


Hence, the number of paths of length 4 from a to d is the (1.4 ith entry of A 4 . Since 


A 4 


'8 0 0 8 ] 
0 8 so! 
0 8 8 0 : 
8 0 0 8j 


there are exactly eight paths of length 4 from a to d. By inspection of the graph, 
we see that a, b, a,b,d\ a, b, a, c,d\ a, b, d, b, d\ a, b. d. c, d; a, c, a. b, d; a. c, a, c, d\ 
a, c. d, b, (/; and a, c, d, c, d are the eight paths from a to d. ■ 


Theorem 2 can be used to find the length of the shortest path between two vertices 
of a graph {see Exercise 32), and it can also be used to determine whether a graph is 
connected (see Exercises 37 and 38). 


Exercises 


1. Does each of the following lists of vertices form a path 
in the following graph':' Which paths are simple? Which 
are circuits? What are the lengths of those that are paths'.’ 
a) u. e. b. c. b b) a, e, a, d, h, c, a 

e) e. h, t2 , d, b, r d) r, b, d. a, e, c 

a b c 



d t 


1. Does cadi of the following lists of vertices form a paih 
in the following graph? Which paths are simple'’ Which 
are circuits? What are the lengths of those that are paths? 
a) a, b. e. c. h b) a.d.a.d.a 

e) a, d, b, e, a d) a, b, e, c, b, d, a 



4- 


In Exercises 3- 5 determine whether the given graph i.s con¬ 
nected. 



/XXX< 



6. How many connected components does each of the 
graphs in Exercises 3 -5 have? For each graph find each 
of its connected components. 

*7. Find the number of paths of length n between two 
different vertices in fd if n is 
a) 2. b) 3. c) 4. d) 5. 

*8. Find the number of paths of length n between any two 
adjacent vertices in Ky for the values of n in 
Exercise 7. 

Find the number of paths of length n between any 
two nonadjacent vertices in X for the values of n in 
Exercise 7. 
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Hi. hind the numberof piths between c and d m the graph 
;it figure I of length 

at 2 b'i 3. cl 4. d) 5. e) 6. f) 7. 

11. f'ind the number of paths from a to e in the directed 
graph in Exercise 1 of length 
a) 2. hi 3. c) 4. d) 3. e) 6. f) 7. 

* 12. Show that a connected graph with n vertices has at least 
« 1 edgc.i. 

13. 1 ft (• ■■■ 1 3i Ki be a simple graph. Let R be the relation 
n ti l'consisting of pairs of vertices («, v) such that there 
is a path from it to r or sued that u ~ v. Show that R is 
ait cquiv alence relation. 

' 14, Show lfi.il ill any .simple graph there is a path from any 
vertex of odd degree In some other vertex of odd degree. 

In Exerciser 13-17 find ail the cut vertices of the given 

graph. 


15 . 




18, Find all the cut edges in the graphs in Exercises 15-17. 

*19, Suppose that v is an endpoint of a cut edge. Prove 
that v is a cut vertex if and only if this vertex is not 
pendant. 

*20. Show that a vertex c in the connected simple graph G 
is a cut vertex if and only if there are vertices u and v, 
both different from c, such that every path between u 
and i‘ passes through c. 

*21. Show 1 that a simple graph with at least two vertices has 
at least two vertices that are not cut vertices. 

*22. Show that an edge in a simple graph is a cut edge if and 
only if this edge is not part of any simple circuit in the 
graph. 

23. A communications lint in a network should be pro¬ 
vided with a backup link if its failure makes it im¬ 
possible for .some message to be sent. For each uf the 
communications networks shown below in (a) and 
[hi. determine those links that should be backed up, 



|yi Burlington Bangor 
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A vertex basis in u directed graph is a set of vertices sudi 
that there is a path to every vertex in the directed graph not 
in the set from some vertex in this set and there is no path 
from any vertex in the set to another vertex in the set. 

24. Find a vertex basis for each of the directed graphs in 
Exercises 7-9of Section 7,2, 

25. What is the significance ut a vertex basis in an influ¬ 
ence graph (described in Example 7 of Section 7.b? 
bind a vertex basis in (he influence graph in this exam¬ 
ple. 

26. Show that if a connected simple graph G is the union 
of the graphs in and O';, then Oi and G have at. least 
otic common vertex. 

*21. Show that if a simple graph (7 has k connected com¬ 
ponents and these components have tii. w;,.... n K ver¬ 
tices. respectively, then the number of edges of G does 
run exceed 

^ f(.H, 2i. 

*28. Use Exercise - ■ to show that a simple graph wish /; 
veniecs and k connected components has ut must i n - 
E Hrr k f 1V2 edges- [Hint: First show- that 

»; ir - {k ■ I %2>i ■ A) 

where n, is (he number of veriiccs in the ith connected 
component.! 


*29. Show that a simple graph G with n vertices is 
connected if it has more than {» - if in - 2j/2 
edges. 

30. Describe the adjacency matrix oi a graph with * con¬ 
nected components when the vertices of the graph are 
listed so that vertices in each connected eamponem arc 
listed smt'syivdv- 

31. How many nontsomorphic connected simple graphs arc 
there with n vertices when >t is 

a) 2? b) 3f e) 4'» d) 57 

32. Explain huw Theorem 2 can be used to litld the 
length of the shortest path from a vertex ; to a 
vertex a- in a graph. 

33. Use Theorem 2 to find the length of the shortest path 
between it and fin the mu digraph in figure 1. 

34. I Ac Theorem 2 to find the length nf the shortest path 
from a to r in the directed graph in Exercise 2 

35. Let P] and P-, he two simple paths between the 
vertices a and i in the simple graph (> that do no: 
contain the same set of edges. Show trial, them is a 
simple circuit in G. 

36. Show that the existence of a simple circuit t>! length 
k, where x i< a positive integer greater than 7, is an 
ireaneiphiv tflvtinMii, 

57, Fxpkin; hew Tiieuiren ? can he usee re determine 

WhCthc e. gliiph is connected 

38. Use Exercise 37 to show that the graph (, re figure 7 
is connected whereas the graph H in that figure a- hot 
cnsineeted 


7.5 

Euler and Hamilton Paths 

INTRODUCTION 

The town of Kbnigsberg. Prussia (.now called Kaliningrad and part of the Russian sc 
Vieb public), was divided into four sections by the brandies of the Preget kiu-r. These torn 
sections included (he two regions cm the hanks of the Preeel. Kneiphof Wand, and the 
region between the two brandies of the Pregei In tite eighteenth century seven bridges 
connected these regions. Figure I depicts ihe.se regions and bridges 

The townspeople took long walks through town on Sunday*. They wondered 
whether ;; was possible to start at some location in the town, travel across ail die 
bridges w-itiwiu crossing any bridge twice, and return io the starting point 

The Swiss maihematidait Leonh a rd Litter solved this problem, llis suiutum. pub¬ 
lished hi 173b. may be the first use oi graph theory. Bttier .studied ibis prt.'blC'n using the 
inulfjgrtipb obtained when the four regions arc represented by vertices and tnbridges 
by edges. This wultigraph is shown in I re:, e 2. 

the problem <rf traveling across every bridge wt (hour cross in g anv bridge ft to re 
diau isneo can be rephrased in terms of this model. The question become:,; is there- a 
sintpic circuit m ibis rilnltigraph that contains every edge? 
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FIGURE I The Seven Bridges 
of Kimigsberg. 


c 



FIGURE t Multigraph Model of 
the Town of Konigsberg. 


DEFINITION 1. An Euler circuit in a graph G is a simple circuit containing every 
edge of G. An Euler path in G is a simple path containing every edge of G, 


The following examples illustrate the concept of Euler circuits and paths. 


FWMPII 1 


Which of the undirected graphs in Figure 3 have an Euler circuit' 1 Of those that do u-.ii. 
which have an Kilter path' 1 


/■ 

A 


X* 

\ 


/ 


\ / 

A' 


/- 


t'i b 

i \ h 


>i i_7 


FIG l ■ R E 3 The Undirected Graphs G\ , Gj, and G%. 


\ 



Leonhard Euler (1707-1783). Leonhard Euler was die son of a Calvinist minister from the vicinity of 
Basel, Switzerland. At 13 he entered the University of Basel, pursuing a career in theology, as his father 
wished. At the university Euler was tutored by Johann Bernoulli of the famous Bernoulli family of mathe¬ 
maticians. Hi;, interest and skills led him to abandon his theological studies and take up mathematics. Elder 
obtained his master's degree in philosophy at the age of 16. In 1727 Peter the Great invited him tn min the 
Academy at St. Petersburg. In 1741 he moved to the Berlin Academy, where he stayed until 17h.fi. He iiieu 
returned to Si. Petersburg, where he remained for the rest of his life. 

Euler was incredibly prolific, contributing to many areas of mathematics, including number theory, 
combinatorics. analysis, as well as its applications to such areas as music and naval architecture. He wrote 
uve r 11 Of)books and papers and left so much unpublished work Ihnl it look 47 years after he died lor all his 
work to he published. During his life his papers accumulated so quickly that he kept alarge pile of articles 
awaiting publication, The Berlin Academy published the papers on lop of this pile so later results were often 
published before results they depended on or superseded. Enter had U children and was able to continue his 
work while a child or (wo bounced on his knees. He was blind fox the last 17 yean of his life, hut hcca.u>t of 
h» fantastic memory this did not diminish his mathematical output. The projeer of publishing his collected 
works, undertaken by the Swiss Society of Natural Science, is still going on ami will require mine ihsu 7' 
volumes. 
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EXAMPLE 2 


a b 



1 1 , H-, H- t 


FIGURE 4 The Directed Graphs H lh Hi, and H 3 . 


Solution: The graph G| has an Euler circuit, for example a, e, c, d, e, b, a. Neither of 
Ihc graphs G~ or G_i has an Euler circuit (the reader should verify this). However, Gj 
has an Euler path, namely, a, t, d, e. h, d, a . h, (h does not have an Euler path (as the 
reader should verify). ■ 


Which of the directed graphs in Figure 4 have an Euler circuit? Of those that do not. 
which have an Euler path? 

Solution: The graph Hi has an Euler circuit, for example, a, g, c, b, g, e, d, f, a. Neither 
H] nor H\ has an Euler circuit (as the reader should verify). Hi has an Euler path, 
namely c, a, b, t\ d. b. but Hi does not (as the reader should verify). I 

NECESSARY AND SUFFICIENT CONDITIONS FOR EULER 
CIRCUITS AND PATHS 


There are simple criteria for determining whether a multigraph has an Euler circuit or 
an Euler path Euler discovered them when he solved the famous Konigsberg bridge 
problem. We will assume that all graphs discussed in this section have a finite number 
of vertices and edges. 

What can we say if a connected multigraph has an Euler circuit? What we can 
shove is that every vertex must have even degree. To do this, first note that an Euler 
circuit begins with a vertex a and continues with an edge incident to a, say {a, b]. The 
edge {a, b\ contributes 1 to deg(n). Each time the circuit passes through a vertex it 
contributes 2 to the vertex's degree, since the circuit enters via an edge incident with 
this vertex and leaves via another such edge. Finally, the circuit terminates where it 
started, contributing 1 to degfa). Therefore, deg(n) must be even, because the circuit 
contributes 1 when it begins, 1 when it ends, and 2 every time it passes through a (if it 
ever does). A vertex other than a has even degree because the circuit contributes 2 to 
its degree each time it passes through tbe vertex. We conclude that if a connected graph 
has an Euler circuit, then every' vertex must have even degree. 

Is this necessary condition for the existence of an Euler circuit also sufficient? That 
is, must an Euler circuitexist in a connected multigraph if all vertices have even degree? 
This question can be settled affirmatively with a construction. 

Suppose that G is a connected multigraph and the degree of every vertex of G is 
even. We wil I form a simple circuit that begins at an arbitrary vertex a of G. Let ip = a. 
First, we arbitrarily chouse an edge {xq, .tj} incident with a. We continue by building a 

simple path {ig, jcj}, {x\, xy\ __ , jc b } that is as long as possible. For instance, in 

the graph G in Figure 5 we begin at a and choose in succession the edges {a, /}, {/, c}, 
{r, /'] and {h. a}. 
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FIGURE 5 Constructing an Euler Circuit in C. 


The path terminates since the graph has a finite number of edges, fi begins at a 
with an edge of the form {«, ,tj, and it terminates at a with an edge of the form {v, a}. 
This follows because each time the path goes through a vertex with even degree, it uses 
only one edge to enter this vertex, so that at least one edge remains for the path to leave 
the vertex. This path may use all the edges, or it may not. 

Art Euler circuit has been constructed if all the edges have been used. Other¬ 
wise, consider the subgraph H obtained from G by deleting the edges already used 
and vertices that are not incident with any remaining edges. When we delete the circuit 
tt, /, r, b, a from the graph in Figure 5, we obtain the subgraph labeled as H. 

Since G is connected, H has at least one vertex in common with the circuit that has 
been deleted. Let w be such a vertex. (In our example, t is (he vertex.) 

Every vertex in H has even degree (because in G all vertices had even degree, and 
for each vertex, pairs of edges incident with this vertex have been deleted to form H). 
Note that H may not he connected. Beginning at tv, construct a simple path in H by 
choosing edges as long as possible, as was done in G, This path must terminate at w. 
For instance, in Figure 5. c. d. e, c is a path in H. Next, form a circuit in G by splicing 
the circuit in // with the original circuit in G (this can he done since w is one of the 
vertices ill this circuit). When this is done in the graph in Figure 5, we obtain the circuit 
a. f, c, J, e, c, b, a. 

Continue this process until all edges have been used, (The process must terminate 
since there are only a finite number of edges in the graph.) This produces an Filler 
circuit. The construction shows that if the vertices of a connected multigraph ail have 
even degree, then the graph has an Euler circuit. 

We summarize these results in Theorem 1. 

A connected multigraph has an Euler circuit if and only if each of its vertices has 
even degree. 


We can now' solve the Konigsbcrg bridge problem. Since the multigraph represent¬ 
ing these bridges, shown in Figure 2, has four vertices of odd degree, it does not have an 
Luier circuit. There is no way to start at a given point, cross each bridge exactly once, 
and return to the starling point. 

Algorithm 1 gives the constructive procedure for finding Euler circuits given in 
the discussion preceding Theorem 1. (Since the circuits in the procedure are chosen 
arbitrarily, there is some ambiguity. We will not bother to remove this ambiguity by 
specifying the steps of the procedure more precisely.) 
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EXAMPLE 3 


| ALGORITHM 1 Constructing Euler Circuits. 

procedure Euter(G: connected multigraph with all vertices of 
even degree) 

circuit : - n circuit in G beginning at an arbitrarily chosen 
vertex with edges successively added to form a path that 
returns to this vertex \ 

H : - G with the edges of this circuit removed 

while H has edges i 

begin I 

I subcircuit : =■ a circuit in H beginning at a vertex in H that • 

! also is an endpoint of an edge of circuit j 

j H: = H with edges of mbcircuit and alI isolated vertices 

j removed 

! circuit : = circuit with mbcircuit inserted at the appropriate j 

vertex j 

end {circuit is an Euler circuit) 

___J 

The next example shows how Euler paths and circuits can be used to solve a type 
of puzzle. 


Many puzzles ask you to draw a picture in a continuous motion without lifting a pencil 
so that no part of the picture is retraced. We can solve such puzzles using Euler circuits 
and paths. For example, can Mohammed’s scimitars, shown in Figure 6, be drawn in 
this way, where the drawing begins and ends at the same point? 

Solution: We can solve this problem since the graph G shown in Figure 6 has an Euler 
circuit. It has such a circuit since all its vertices have even degree, Wc will use Algo¬ 
rithm I to construct an Euler circuit. First, we form the circuit a, h, d. c, b, e. i, f, e, a. 
We obtain the subgraph H by deleting the edges in this circuit and all vertices that be¬ 
come isolated when these edges are removed. Then we form the circuit d, g, h, j , i, h. k, 
g t f. d in H. After forming this circuit we have used all edges in G. Splicing this 
new circuit into the first circuit at the appropriate place produces the Euler circuit 
a, b, d, g, h, j, i, h, k, g, f, d, c, b, e, i, f, e, a. This circuit gives a way to draw the scim¬ 
itars without lifting the pencil or retracing part of the picture. ■ 



FIGURE 6 Mohammed’s Scimitars. 
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EXAMPLE 4 


Another algorithm for constructing Euler circuits, called Fleury’s algorithm, is de¬ 
scribed in the exercises at the end of this section. 

We will now show that a connected multigraph has an Euler path (and not an Eu¬ 
ler circuit) if and only if it has exactly two vertices of odd degree. First, suppose that 
a connected mulrigraph does have an Euler path from a to b, but not an Euler circuit. 
The first edge of the path contributes 1 to the degree of a. A contribution of 2 to the 
degree of a is made ever)' time the path passes through a. The last edge in the path con¬ 
tributes 1 to the degree of b. Every time the path goes through b there is a contribution 
of 2 to its degree. Consequently, both a and b have odd degree. Every other vertex has 
even degree, since the path contributes 2 to the degree of a vertex whenever it passes 
through it. 

Now consider the converse. Suppose that a graph has exactly two vertices of odd 
degree, say a and b. Consider the larger graph made up of the original graph with the 
addition of an edge {a, b}. Every vertex of this larger graph has even degree, so that 
there is an Euler circuit. The removal of the new edge produces an Euler path in the 
original graph, The following theorem summarizes these results. 

A connected multigraph has as Euler path but not aa Euler circuit if and only if it 
has exactly two vertices of odd degree. 


Which graphs shown in Figure 7 have an Euler path? 

Solution: G\ contains exactly two vertices of odd degree, namely, b and d. Hence, 
it has an Euler path that must have b and d as its endpoints. One such Euler path is 
d. a, b, c, d, b. Similarly, (h has exactly two vertices of odd degree, namely, b and d. 
So it has an Euler path that must have h and / as endpoints. One such Euler path is 
b. a, g, /, e, d, c g, b, c, f. d. GA has no Euler path since it has six vertices of odd degree. 


Returning to eighteenth-century' Konigsberg, is it possible to start at some point 
in the town, travel across all the bridges, and end up at some other point in town? 
This question can be answered by determining whether there is an Euler path in the 
multi graph representing the bridges in Konigsberg. Since there are four vertices of odd 
degree in this multigraph, there is no Euler path, so such a trip is impossible. 

Necessary and sufficient conditions for Euler paths and circuits in directed graphs 
are discussed in the exercises at the end of this section. 
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HAMILTON PATHS AND CIRCUITS 


We have developed necessary and sufficient conditions for the existence of paths and 
circuits that contain every edge of a multigraph exactly once. Can we do the same for 
simple paths and circuits that contain every vertex of the graph exactly once ’ 


DEFINITION 2. A path Jt&, xy, ..., x B to the graph G — (V,E) is called a 
Hamilton path if V = {.*& jcj. x„-j, x«}xod x ( # xj for 0 s i < j s n. A cir¬ 

cuit x ir ,,. 7 x K -i,x n ,xa (with n> I)ma g^pb G - (V, 8 )&iaHed a Hampton 
circuit if Xft, x\ t ..x„~\, x, is a Hamilton path. 


This terminology comes from a puzzle invented in 1857 by the Irish mathematician 
Sir William Rowan Hamilton. Hamilton’s puzzle consisted of a wooden dodecahedron 
[a polyhedron with 12 regular pentagons as faces, as shown in Figure Start, with a peg 
at each vertex of the dodecahedron, and string. The 20 vertices of the dodecahedron 
were labeled with different cities in the world. The obiect of the puzzle was to start at a 
city and travel along the edges of the dodecahedron, visiting each of the other 19 cities 
exactly once, and end back at the first city. The circuit traveled was marked off using 
the strings and pegs. 

Since the author cannot supply each reader with a wooden solid with pegs and 
string, we will consider the equivalent question: Is there a circuit in the graph shown 
in Figure 8(b) that passes through each vertex exactly once? This solves the puzzle 
since this graph is isomorphic lo the graph consisting of the vertices and edges of the 
dodecahedron. A solution of Hamilton’s puzzle is shown in Figure 9. 


Wi‘b 


William Rowan Hamilton I. S 805-1865), Witliam Rowan Hamilton, the most famous Irish scientist ever 
to have lived, was torn in 1805 in Dublin. His father was a successful lawyer, his mother tame from a family 
noted for their intelligence, and he was a child prodigy, By the age of 3 he was an excellent reader and had 
mastered advanced arithmetic. Because of his brilliance, he wassent off to live with his uncle James, a noted 
linguist. By age 8 Hamilton had learned Latin, Greek, and Hebrew; by 10 he had also learned Italian and 
French and he began his study of oriental languages, including Arabic, Sanskrit, and Persian. During this 
period he took pride in knowing as many languages as his age. At 17, no longer devoted to learning new lan¬ 
guages and having mastered calculus and much mathematical astronomy, he began original work in optica, 
and he also found an important mistake in Laplace’s work on celestial mechanics. Before entering Trinity 
College, Dublin, at 18, Hamilton had not attended school; rather, he received private tutoring. Ai Trinity, 
he was a superior student in both the sciences and the classics. Prior to receiving h is degree, because of his 
brilliance he was appointed the Astiomtmer Royal of Ireland, beating out several famous astronomers for 
the post. Ho held this position until his death, living and working at Dunsink Observatory outside of Dublin. 
Hamilton made important contributions to optics, abstract algebra, and dynamics. Hamilton invented alge¬ 
braic objects called quaternions as an example of a noneommutative system. He discovered the appropriate 
way to multiply quaternions while walking along a canal in Dublin, tn his excitement, he carved the for¬ 
mula in the stone of a bridge crossing (he canal, a spot marked today by a plaque. Later, Hamilton remained 
obsessed with quaternions, working to apply them to other areas of mathematics, instead of moving to new 
areas of research. 

In ! 857 Hamilton invented "The icosian Game" based on his work in noneommutative algebra. He 
sold the idea for 25 pounds in a dealer in games and puzzles. (Since the game never sold well, this turned 
nut to be a bad investment tor the dealer,) The “Traveler’s Dodecahedron,” also called “A Voyage Round 
the World,” the puzzle described in this section, is a variant of (hat game. 

Hamilton married his third love in 1833, but his marriage worked out poorly, since his wife, a semi- 
invalid, was unable to cope with his household affairs. He suffered from alcoholism and lived reclusively for 
the last two decades of his life. He died from gout in 1865, leaving masses of papers containing unpublished, 
research- Mixed in with these papers were a large number of dinner plates, many containing the remains of 
desiccated, uneaten chops. 
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World ’ 7 Puzzle. 


I \ Wll’l J 5 Which of ihc simple graphs in figure 10 have a Hamilton circuit or, if not, a Hamilton 

palin’ 

Saiutiem: (?i has a Hamilton circuit: a. h, c,d, e, a. There is no Hamilton circuit in Cn 
( this can be seen by noting that any circuit containing every vertex must contain Lhe 
edge b) twice), but Gj does have a Hamilton path, namely a, b, c, d. Cn, has neither 
a Hamilton circuit nor a Hamilton path, since any path containing all vertices must 
contain one of the edges {«, b\. {e. /}. and {e, d} more than once, ■ 


Is there a simple way to determine whether a graph has a Hamilton circuit or path? 
At first, it might seem that there should he an easy way to determine this, since there 
is a simple way to answer the similar question of' whether a graph has an Euler cir¬ 
cuit. Surprisingly, there are no known simple necessary and sufficient criteria for the 
existence of Hamilton circuits. However, many theorems are known that give sufficient 
conditions for the existence of Hamilton circuits. Also, certain properties can be used to 
show that a graph has no Hamilton circuit. For instance, a graph with a vertex of degree 
1 cannot have a Hamilton circuit, since in a Hamilton circuit each vertex is incident 
w ith two edges in the circuit. Moreover, if a vertex in the graph has degree 2, then both 
edges that are incident with this vertex must be part of any Hamilton circuit, Also, note 
that when a Hamilton circuit is being constructed and this circuit has passed through 
a vertex, then all remaining edges incident with this vertex, other than the two used in 
;he circuit, can be removed from consideration. Furthermore, a Hamilton circuit cannot 
contain a smaller circuit within it. 



(a f(. 

FIGURE U) Three Simple Graphs. 
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EXAMPLE 


EXAMPLE 


THEOREM 


EXAMPLE S 



Fit! ERE 1! Two Graphs That Do Not Ha\ e a Hamilton Circuit. 


Show that neither graph displayed in Figure 11 has a Hamilton circuit. 

Solution: There i.s no Hamilton circuit in O' since fir’ has a vertex of degree 1. namely, e. 

Now consider H. Since the degrees oi the vertices a, h, d. and e are all 2, every 
edge incident with these vertices must be part of any Hamilton circuit. It is now easy 
to sec that no Hamilton circuit can exist in H, for any Hamilton circuit would have to 
contain four edges incident with <*, which is impossible. ■ 


Show that K„ lias a Hamilton circuit whenever n ■'? 3. 

Solution: We can form a Hamilton circuit in K a beginning at any vertex. Such a circuit 
can be built by visiting vertices in any order we choose, as long as the path begins and 
ends at the same vertex and visits each other vertex exactly once. This is possible since 
there are edges in K„ between any two vertices. ■ 


We now state a theorem that gives sufficient conditions for the existence of Hamil¬ 
ton circuits. This i.s just one of many such theorems known. 

1 Il'G is a connected simple graph with n vertices where n £ 3, then G has a Hamilton 

circuit if the degree of each vertex is at least nJ2, 

The best algorithms known for finding a Hamilton circuit in a graph or determining 
web that no such circuit exists have exponential worst-case lime complexity (in the number 
of vertices of the graph). Finding an algorithm that solves this problem with polynomial 
worst case time complexity would be a major accomplishment, since the existence of 
such an algorithm would imply that many other seemingly intractable problems could 
be solved using algorithms with polynomial worst-case time complexity. 

We will now give an application of Hamilton circuits to coding. 


C!m\ Codes The position of a rotating pointer can be represented in digital form. One 
way to do this is to split the circle into 2" arcs of equal length and to assign a bit string 
of length n to each arc. Two ways to do this using hit strings of length three are shown 
in Figure 12. 

The digital representation of the position of the pointer can be determined using a 
set or n contacts. Each contact is used to read one bit in the digital representation of the 
position This is illustrated in Figure 13 for the two assignments from Figure 12. 

When the pointer is near the boundary of two arcs, a mistake may be made 
in reading its position. This may result in a major error in the bit string read. For 


PDF created with pdfFactory trial version www.pdffactorv.com 










7 / Graphs 


484 



FIGURE 12 Converting the Position of a Pointer into Digital Form, 


Third bit is 1 here 



First bit is I here 



FIGURE 13 The Digital Representation of the Position of the Pointer. 
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FIGURE 14 A Hamilton Circuit for Q\, 


instance, in the coding scheme in Figure 12(a). if a small emir is made in determin¬ 
ing the position of the pointer, the bit string I(X) is read instead of Oil, All three 
hits are incorrect! To minimize the effect of an error in determining the position of 
the pointer, the assignment of the bit strings to the 2" arcs should be made so that 
only one bit is different in the bit strings represented by adjacent arcs. This is ex¬ 
actly the situation in the coding scheme in Figure 12(b). .An error in determining 
the position of the pointer gives the bit string 010 instead of Oil, Only one bit is 
wrong, 

A Gray code is a labeling of the arcs of the circle so that adjacent arcs are la¬ 
in® beled with bit strings that differ in exactly one bit. The assignment in Figure 12(b) 
is a Gray code. We can find a Gray code by listing all bit strings of length n in such 
a way that each string differs in exactly one position from the preceding bit string, 
and the last string differs from the first in exactly one position. We can model this 
problem using the n-cubc Q„. What is needed to solve this problem is a Hamil¬ 
ton circuit in Q r ,. Such Hamilton circuits are easily found. For instance, a Hamil¬ 
ton circuit for Q% is displayed in Figure 14, The sequence of bit strings differing in 
exactly one bit produced by this Hamilton circuit is 000, 001, Oil, 0)0, ill), 111, 
101 , 100 , 

Gray codes are named after Frank Gray, who invented them in the 1940s at 
AT&T Bell Laboratories to minimize the effect of errors in transmitting digital 

signals. ■ 


Exercises 
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8. Deicmiuie whether the graph in Exercise 1 has an Euler 
path, ('(instruct such a path if it exists. 


9, Determine whether the graph in Exercise 1 has an Euler 
path. Construct such a path ii it exists. 

10. Determine whether the graph m Exercise 3 has an Euler 
path. Construct such a path if it exists. 

11. Determine whether the graph i n Exercise 4 has an Euler 
path. Construct such a path if it exists. 

12. Determine whether the graph in Exercise 5 has an Euler 
path. Construct such a path if it exists. 

13. Determine whether the graph in Exercise 6has an Euler 
path. Construct such a path if it exists. 

14. Determine whether the graph in Exercise 7 has an Euler 
path. Construct such a path if it exists. 

15. hi Kaliningrad (.the Russian name for Kontgsberg) 
there are two additional bridges, besides the seven that 
were present in the 18th century. These new bridges 
connect regions 8 and C and regions B and D f respec¬ 
tively. Can someone cross all nine bridges in Kalin¬ 
ingrad exactly once and return to the starting point? 

16. Can someone cross all the bridges shown in the follow¬ 
ing map exactly once and return to the starting point? 



17, When can the center lines of the streets in a city be 
painted without traveling a street more than once? (As¬ 
sume that all the streets are two-way streets.) 

18. Devise a procedure, similar to Algorithm 1, for con¬ 
structing Euler paths in rnultigraphs. 

In Exercises 19 21 determine whether the picture shown 
can be drawn with a pencil in a continuous motion without 
lifting the pencil or retracing part of the picture. 


19. A 

-—/—\ v 
V V 
c_ v .A 


20 . 
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21 . 



*22. Show that a directed multigraph having no isolated ver¬ 
tices has an Euler circuit if and only if the graph is 
weakly connected and the in-degree and out-degree of 
each vertex are equal. 

*23, Show that a directed multi graph having no isolated ver¬ 
tices has an Euler path but not tut Euler circuit if and 
only if the; graph is weakly connected and the in-degree 
and out-degree of each vertex are equal for all but two 
vertices, one that has in-degree 1 larger than its ont- 
degree and the oilier that has out-degree 1 larger than 
its in-degrcc. 

In Exercises 2*1-28 determine whether the directed graph 
shown has an Euler circuit, Construct an Euler circuit if it 
exists. 



26, a 


I //t I 

- * 

J 


27. a 


/ i 


. t / 


/ / 


/ 


he¬ 
ll ■■■ 





29. Determine whether the directed graph in Exercise 24 
has an Euler path. Construct an Euler path if one exists. 

30. Determine whether the directed graph in Exercise 25 
has an Euler path. Construct an Euler path if one exists, 

31. Determine whether the directed graph in Exercise 26 
has an Euler path. Construct an Euler path if one exists. 

32. Determine whether the directed graph tit Exercise 27 
has an Euler path. Construct an Euler path if one exists. 

33. Determine whether the directed graph in Exercise 
has an Euler path. Construct an Euler path if one exists, 

*34, Devise an algorithm for constructing .Euler circuits in 
directed graphs. 

35. Devise an algorithm for constructing Euler paths in di¬ 
rected graphs, 

36. For which values of n do the following graphs have an 
Euler circuit !' 

st) K. bi ( c) H', d: Q„ 

37. For which values ofn do the graphs in Exercise 36 have 
an Euler path but no Euler circuit? 

3S. For which values of ,>?) and a does the complete bipartite 
graph K mn have an 

a) Euler circuit 2 

b) Euler path? 

39. Find the least number of times it is necessary to lift 
a pencil from the paper when drawing each of the 
graphs in Exorcises 1-7 without retracing any pan of 
the graph. 

Jr. Exercises 40-46 determine whether the given graph has 
a Hamilton circuit. If it does, find such a circuit. If it does 
not, give an argument to show why no such circuit exists. 

40. a c! 

N x 



h 


41, 
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*5), Does the graph in Exercise 44 have a Hamilton path? 
If so, find such a path. If it does not, give an argument 
to show why no such path exists. 

52. Does the graph in Exercise 45 have a Hamilton path? 
If so, find such a path. If it does not, give an argument 
to show why no such path exists. 

*53. Does the graph in Exercise 46 have a Hamilton path? 
If so, find such a path. If it does not, give an argument 
to show why no such path exists. 

54. For which values of n do the graphs in Exercise 36 have 
a Hamilton circuit? 

55. For which values of m and n does the complete bipartite 
graph „ have a Hamilton circuit? 

*56, Show' that the Petersen graph, shown in the following 
diagram, does not have a Hamilton circuit, hut that the 
subgraph obtained by deleting a vertex v, and all edges 
incident with v, docs have a Hamilton circuit. 


a 



*57. Show that there is a Gray code of order n whenever n 
is a positive integer, or equivalently, show' that the n- 
eube Q„, n > 1, always has a Hamilton circuit. (Hint: 
Use mathematical induction. Show how to produce a 
Gray code of order n from one of order n - 1,) 


46. a b 



47, Does the graph in Exercise 40 have a Hamilton path? 
If so, find such a path. If it does not, give an argument 
to show why no such path exists, 

48. Does the graph in Exercise 41 have a Hamilton path? 
If so, find Such a path. If it does not, give an argument 
to show why no such path exists. 

49. Does the graph in Exercise 42 have a Hamilton path? 
If so, find such a path. If it does not, give an argument 
to show why no such path exists. 

50, Does the graph in Exercise 43 have a Hamilton path? 
If so, find such a path. If it does not, give an argument 
to show why no such path exists. 


Fleury’s algorithm for constructing Euler circuits begins 
with an arbitrary vertex of a connected multigraph and 
forms a circuit by choosing edges successively. Once an 
edge is chosen, it is removed. Edges are chosen succes¬ 
sively so that each edge begins where the last edge ends, 
and so that this edge is not a cut edge unless there is no 
alternative. 

58, Use Fleur, -, algorithm to find an Euler circuit in the 
graph G in Example 5. 

*59, Express Fleury’s algorithm in pseudocode. 

**6fi. Prove that Fleury’s algorithm always produces an Euler 
circuit. 

*61. Give a variant of Fleury’s algorithm to produce Euler 
paths. 

62. A diagnostic message can be sent out over a computer 
network to perform tests over all links and in all de¬ 
vices. What sort of paths should be used to test all 
links? To test all devices? 

63, Show that a bipartite graph with an odd number of ver¬ 
tices does not have a Hamilton circuit. 
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A knight is a chess piece that can move cither two spaces 
horizontally and one space vertically or one space hor¬ 
izontally and two spaces vertically. That is, a knight on 
square u, v) can move to any of the eight squares (x r 2, 
yt ! 1. |.t ± 1, v ± 2), if these square s are on the chessboard, 
as illustrated below. 



^ A knight's tour is a sequence of legal moves by a knight 
starting at some square and visiting each square exactly 
once. A knight’s tour is called reentrant if there is a legal 
move that titkes the knight from the last square of die tour 
back to where the tour began. We can model knight’s tours 
using the graph that has a vertex for each square on the board, 
with an edge connecting two vertices if a knight can legally 


move between the squares represented by these vertices. 

64. Draw the graph that represents the legal moves of a 
knight on a 3 X 3 chessboard. 

65. Draw the graph that represents the legal moves of a 
knight on a 3 x 4 chessboard. 

66. a) Show that finding a knight’s tour on an mx n chess¬ 

board is equivalent to finding a Hamilton path on 
die graph representing the legal moves of a knight 
on that board, 

b) Show that finding a reentrant knight's tour on an 
m x n chessboard is equivalent to finding a Hamil¬ 
ton circuit on the corresponding graph. 

*67. Show that there is a knight’s tour on a 3 x4 chessboard. 

*6if, Show' that there is no knight’s tour on a 3X3 chess¬ 
board. 

*69, Show that there is no knight’s tour on a 4x4 chess¬ 
board. 

70. Show that the graph representing the legal moves of a 
knight on an m X n chessboard, whenever m and n are 
positive integers, is bipartite. 

71. Show that there is no reentrant knight’s tour on an m x 
n chessboard when m and n are both odd, (Hint: Use 
Exercises 63, 66b, and 70.j 

*72. Show that there is a knight’s tour on an 8x8 chess¬ 
board. ( Hint: You. can construct a knight's tour using 
the following method invented by Warnsdorff. Start 
in any square, and then always move to a square 
connected to the fewest number of unused squares. 
Although this method may not always produce a 
knight’s tour, it often dues.) 


mb 


Jut)us Peter Christian Petersen I 1839-1 9 Mi). Julius Petersen was bom in the Danish town of Sore 
His father was a dyei. In 1854 his parents were no longer able to pay for his schooling, st> he became an 
apprentice in an uncle's grocery store. When this uncle died, he left Petersen enough money to return to 
school. After graduating, he began studying engineering at the Polytechnics! School in Copenhagen, later 
deciding to concentrate on mathematics. He published his first textbook, a book on logarithms, in 1358. 
When his inheritance ran out, he had to teach to make a living. From 1859 until 1871 Petersen taught 
at a prestigious private high school in Copenhagen. While teaching high school he continued his studies, 
entering Copenhagen University in 1862. He married Laura Rertelsen in 1862: they had three children, two 
sons and a daughter, 

Petersen obtained a mathematics degree from Copenhagen University in 1866 and finally obtained his 
doctorate in 1871 from that school, After receiving fits doctorate, he taught at a polytechnic and military 
academy In 1887 he was appointed to a professorship at the University of Copenhagen. Petersen was well- 
known m Denmark as the author of a large: series of textbooks for high schools and universities. One of his 
books. Methods and Theories for the Solution of Problems of Geometrical Construction, was translated into 
eight languages, with the English language version last reprinted in I960 and the French version reprinted 
at; recently as 1990, more than a century after the original publication date 

Petersen worked in a wide range of areas, including algebra, analysis, cryptography, geometry', me¬ 
chanics, mathematical economics, ami number theory. His contributions to graph theory, including results on 
regular graphs, are his best-known work. He was noted fur bis clarity of exposition, problem-solving skills, 
originality, sense of humor, vigor, and teaching. One interesting fact about Petersen was that he preferred 
not to read the writings of other mathematicians. This led him often to rediscover results already proved by 
others, often with embarrassing consequences. However, he was often angry when other mathematicians 
did not read his writings! 

Petersen's death was front page news in Copenhagen. A newspaper of the lime described him as the 
Hans Christian Andersen of science—a child of the people who made good in the academic world. 
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7.6_ _ __ ___ 

Shortest Path Problems 

INTRODUCTION 

Many problems can be modeled using graphs with weights assigned to their edges. 
As an illustration, consider how an airline system can be modeled. We set up the ba¬ 
sic graph model by representing cities by vertices and flights by edges. Problems in¬ 
volving distances t.tn be modeled by assigning distances between cities to the edges. 
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FIGURE 1 Weighted (Graphs Modeling an Airline System. 
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FIGURE 2 Weighted Graphs Modeling a Computer Network. 


Problems involving flight time can be modeled by assigning flight times to edges. Prob¬ 
lems involving fares can be modeled by assigning fares to the edges. Figure 1 displays 
three different assignments of weights to the edges of a graph representing distances, 
flight times, and fares, respectively. 

Graphs that have a number assigned to each edge are called weighted graphs. 
Weighted graphs are used to model computer networks. Communications costs (such 
as the monthly cost of leasing a telephone line), the response times of the computers 
over these lines, or the distance between computers, can all be studied using weighted 
graphs. Figure 2 displays weighted graphs that represent three ways to assign weights 
to the edges of a graph of a computer network, corresponding to costs, response times 
over the lines, and distance. 

Several types of problems involving weighted graphs arise frequently. Determining 
the path of least length between two vertices in a network is one such problem. To be 
more specific, let the length of a path in a weighted graph be the sum of the weights 
of the edges of this path. (The reader should note that this use of the term length is 
different from the use of length to denote the number of edges in a path in a graph 
without weights.) The question is: What is the shortest path, that is, the path of least 
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length, between two given vertices? For instance, in the airline system represented by 
the weighted graph shown in Figure 1. what is the shortest path in air distance between 
Boston and Los Angeles? What combinations of fl ights has the smallest total flight time 
(that is, total time in the air, not including time between flights) between Boston and Los 
Angeles? What is the cheapest fare between these two cities? In the computer network 
shown in Figure 2, what is the least expensive set of telephone lines needed to connect 
the computers in San Francisco with those in New York? Which set of telephone lines 
gives the fastest response time for communications between San Francisco and New 
York? Which set of lines has the shortest overall distance? 

Another important problem involving weighted graphs asks for the circuit of short¬ 
est total length that visits every vertex of a complete graph exactly once. This is the 
famous traveling salesman problem which asks for the order a salesman should visit 
each of the cities on his route exactly once so that he travels the minimum total distance. 
We will discuss the traveling salesman problem later in this section. 


A SHORTEST PATH ALGORITHM 


There are several different algorithms that And the shortest path between two vertices in 
wet a weighted graph. We will present an algorithm discovered by the Dutch mathematician 
E. Dijkstra in 1959. The version we will describe solves this problem in undirected 
weighted graphs where all the weights are positive. It is easy to adapt it to solve shortest 
path problems in directed graphs. 

Before giving a formal presentation of the algorithm, we will give a motivating 
example. 


EXAMPLE 1 What is the length of the shortest path between a and z in the weighted graph shown in 

Figure 3? 

Solution: Although the shortest path is easily found by inspection, we will develop 
some ideas useful in understanding Djjkstra's algorithm. We will solve this prob¬ 
lem by finding the length of the shortest path from a to successive vertices, until ; 
is reached. 

The only paths starting at a that contain no vertex other than a (until the termi¬ 
nal vertex is reached) are a. b and a, d. Since the lengths of a, b and a, d are 4 and 2, 
respectively, it follows that d is the closest vertex to a. 

We can find the next closest vertex by looking at all paths that go through only a 
and d (until the terminal vertex is reached). The shortest such path to b is still a, h, with 


Edsger Hybe Dijkstra (bnrn l'J30i. Edsgcr Dijkstra, born in the Netherlands, began programming enm- 
W(b P ulws ,n the wdy 1950s while studying theoretical physics at the University of Leiden, in 1952. realizing 
that he was more interested in programming than in physics, he quickly completed the requirements for 
his physics degree and began his career as a programmer, even though programming was not a recognized 
profession. (In 1957, the authorities in Amsterdam refused to accept “programming” as his profession on 
his marriage license. However, they did accept ‘theoretical physicist” when he changed his entry to this, j 
Dijkstra has beer, one of the most forceful proponents of programming as a scientific discipline. He has 
made fundamental contributions to the areas of operating systems, including deadlock avoidance; program 
mttig languages, including the notion of structured programming; and algorithms. In 1972 Di jkw a received 
the Turing Award from the Association for Computing Machinery, one of the most prestigious awards in 
computer science. Dijkstra became a Burroughs Research Fellow ill 1972, and in 1984 lie was appointed to 
a chair in Computer Science al the University of Texas, Austin. 
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FIGl'RE 3 A Weighted Simple Graph. 


length 4, and the shortest such path to e is a. d, e, with length 5. Consequently, the next 
closest vertex to ci is b 

To find the third closest vertex to a, we need to examine only paths that go through 
only a, d, and b (until the terminal vertex is reached). There is a path of length 7 to c, 
namely, a , b. c, and a path of length 6 to z, namely, a, d. e. z. Consequently, z is the next 
closest vertex to a , and the length of the shortest path to z is 6. ■ 


f’xample ! illustrates the general principles used in Dijkslra’s algorithm. Note that 
she shortest path from a to z could have been found by inspection. However, inspec¬ 
tion is impractical for both humans and computers for graphs with large numbers 
ot edges. 

We will now consider the general problem of finding the length of the shortest path 
between a and z in an undirected connected .simple weighted graph, Dijkstnfs algorithm 
proceeds by finding the length of the shortest path from a to a first vertex, the length of 
the shortest path from a to a second vertex, and so on, until the length of the shortest 
path from a to z is found. 

The algorithm relies on a series of iterations. A distinguished set of vertices is 
constructed by adding one vertex at each iteration, A labeling procedure is carried out 
at each iteration. In this labeling procedure, a vertex w is labeled with the length of 
the shortest path from a to w that contains only vertices already in the distinguished 
set. The vertex added to the distinguished set is one with a minimal label among those 
vertices not already in the set. 

We now give the details of Dijkslra’s algorithm. It begins by labeling a with 0 and 
the other vertices with We use the notation Lq(o) = 0 and Lq(v) = ® for these labels 
before any iterations have taken place (the subscript 0 stands for the “Oth” iteration). 
These labels axe the lengths of the shortest paths from a to the vertices, where the paths 
contain only the vertex a. (Since no path from a to a vertex different from a exists, »is 
the length of the shortest path between a and this vertex.) 

Dijkstra’s algorithm proceeds by forming a distinguished set of vertices. Let S k 
denote this set after k iterations of the labeling procedure. We begirt with Sq = 0. The 
set 5* is formed from 5*_j by adding a vertex u not in S k -1 with the smallest label, 
Once u is added to S t , we update the labels of all vertices not in S k . so that L l: (v), the 
label of the vertex v at the Mi stage, is the length of the shortest path from a to v that 
contains vertices only in 5- (that is, vertices that were already in the distinguished set 
together with u). 

Let v be a vertex not in S k . To update the label of i\ note that Lis the length 
of the shortest path from a to v containing only vertices in S k . The updating can be 
carried out efficiently when the following observation is used: The shortest path from a 
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i w\im : 


to v containing only elements of $ k is either the shortest path from a to v that contains 
only elements of (that is, the distinguished vertices not including «), or it is the 
shortest path from a to u at the (k ■ -1 )st stage with the edge (tt, v) added. In other words, 

L^a, v) - min /, ( ](o, v), /.<_](«. u) win. v)}. 

Phis procedure is iterated by successively adding vertices to the distinguished set 
until ~ is added. When ,7 is added to the distinguished set, its label is the length of the 
shortest path from a to z- Dijkstra’s algorithm is given in Algorithm 1. Later we will 
give a proof that this algorithm is correct. 


| .u.coumni 1 Dijkslra’.s Algorithm. 

procedure DijkstniiG: weighted connected simple graph, with 
j all weights positive) 

: {G has vertices a - 1 % v,.v„ - z and weights wfuj, vj) 

j where w(Vj. vj) — K if {vy, vis not an edge in C} 

j for 1 := 1 to n 

I L{et): =- 0 

S : - 0 

; {the labels are now' initialized so that the label of a is zero and all 
other labels arc *, and S is the empty set) 
j while z$.S 
begin 

| if:-- a vertex not in S with Liu) minimai 

1 i T :=.?U{«} 

| for all vertices v not in S 

if L(i r) 4- win, v) < Ltv) then Uv): — Liu ) 4- w(u, v) 

I {this adds a vertex to S with minimai label and updates the 

! labels of vertices not in .S’) 


1 end {Lu) - length of shortest oath from a to z\ 



The following example illustrates how Dijkstra’s algorithm works. Afterward, we 
will show that this algorithm always produces the length of the shortest path between 
two vertices in a weighted graph. 


Use Dijkstra’s algorithm to find the length of the shortest path between the vertices a 
and : in the weighted graph displayed in Figure 4(a). 

Snhition: The steps used by Dijkstra’s algorithm to find the shortest path between a 
and 0 are shown in Figure 4. At each iteration of the algorithm the vertices of the set 
circled. The shortest path from a to each vertex containing only vertices in is 
indicated for each iteration. The algorithm terminates when \ is circled. We find that 
the shortest path from .a to 7 is a, c, b. d e, 7 , with length 13. ■ 

Remark: in performing Dijkstra's algorithm it is sometimes more convenient to keep 
track of labels of vertices in each step using a table instead qf redrawing the graph for 
each step. 
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4»o <i« b 4kd (J<*> b 3 k;. c j J ]0i.a, lI 



FI (JURE 4 Using Dijkstra’s Algorithm to find the Shortest Path from a to z. 


Next, we use an inductive argument to show that Di jkstra’.s algorithm produces the 
length of the shortest path between two vertices a and r in an undirected connected 
weighted graph. Take as ihe induction hypothesis the following assertion: At the Jtth 
iteration 

(i) the label of a vertex v, v ^ 0, in S is the length of the shortest path from a to 
this vertex, and 

(ii) ihe label of a vertex not in S is the length of the shortest path from a to this 
vertex that contains only (besides the vertex itself) vertices in .V. 

When k - 0, before any iterations are carried out, S - {a}, so the length of the shortest 
path from a to a vertex other than a is w, and the length of the shortest path from a to 
itself is 0 (here we are allowing a path to have no edges in it). Hence, the basis case is 
true. 

Assume that the inductive hypothesis holds for the Ath iteration. Let v be the vertex 
added to S at the (A + l)st iteration so that v is a vertex not in S at the end of the Ath 
iteration with the smallest label (in the case of ties, any vertex with smallest label may 
be used). 

From the inductive hypothesis we sec that the vertices in S before the (A + l ist 
iteration are labeled with the length of Ihe shortest path from a, Also, v must be labeled 
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THEOREM 1 

with the length of the shortest path to it from a. If this were not the case, at the end of the 
Eth iteration there would be a path of length less than L k (v) containing a vertex not in S 
(because LiSy) is the length of the shortest path from a to v containing only vertices in 
S after the kth iteration). Let u be the first vertex not in S in such a path. There is a path 
with length less than L k (v) from a to u containing only vertices of S. This contradicts 
the choice of v. Hence, (i) holds at the end of the (k + l)st iteration. 

Let u he a vertex not in S after k + i iterations. A shortest path from a to u containing 
only elements of S either contains v or it does not. If it does not contain v, then by the 
inductive hypothesis its length is Z,*{a). If it does contain v, then it must be made up 
of a path from <3 to v of shortest possible length containing elements of S other than v, 
followed by the edge from v to u. In this case its length would be Lt(v) + w(v, a). This 
shows that (it) is true, since £* +1 (m) = min{£*(«), L k (y) + w(v, u}}. 

The following theorem has been proved. 

Dijkstra’s algorithm finds the length of a shortest path between two vertices in a 
connected simple aedtreeted weighted graph. 

THEOREM 2 

We can now estimate the computational complexity of Dijkstra’s algorithm (in terms 
of additions and comparisons). The algorithm uses no more than n - 1 iterations, since 
one vertex is added to the distinguished set at each iteration. We are done if we can 
estimate the number of operations used for each iteration. We can identify the vertex 
not in St w ith the smallest label using no more than n - 1 comparisons. Then we use an 
addition and a comparison to update the label of each vertex not in S k . It follows that no 
more than 2{n - 1) operations are used at each iteration, since there are no more than 
n -1 labels to update at each iteration. Since we use no more than n - 1 iterations, each 
using no more than 2(n - 1) operations, we have the following theorem, 

Dijkstra’s algorithm ases Of'it 2 ) operations (additions and cmapanwosjto find the 
length of die shortest path between two vertices in a connected Kitspfc undirected 
weighted graph. 

w4b 

THE TRAVELING SALESMAN PROBLEM 

We now discuss an important problem involving weighted graphs. Consider the fol¬ 
lowing problem: A traveling salesman wants to visit each of n cities exactly once and 
return to his starting point. For example, suppose that the salesman w ants to visit De¬ 
troit, Toledo, Saginaw, Grand Rapids, and Kalamazoo (see Figure 5). In which order 
should he visit these cities to travel the minimum total distance? To solve this problem 
we can assume the salesman starts in Detroit f since this must be part of the circuit) and 
examine all possible ways for hirn to visit the oilier four cities and then return to Detroit 
(starting elsewhere will produce the same circuits). There are a total of 24 such circuits, 
but since we travel the same distance when we travel a circuit in reverse order, we need 
only consider 12 different circuits to find the minimum total distance he must travel. 
We list these 12 different circuits and the total distance traveled for each circuit. As 
can be seen from the list, the minimum total distance of 458 miles is traveled using the 
circuit Detroit-To 1 edo—Kalamazoo—Grand Rapids-Saginaw-Detroit (or its reverse). 

We just described an instance of the traveling salesman problem. The traveling 
salesman problem asks for the circuit of minimum total weight in a weighted, complete, 
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FIGURE 5 Tile Graph Showing the Distances Between Five Cities. 


undirected graph that visits each vertex exactly once and returns to its starting point. 
This is equivalent to ashing for a Hamilton circuit with nunimum total weight in the 
complete graph, since each vertex is visited exactly once in the circuit. 

The most straightforward way to solve an instance of the traveling salesman prob¬ 
lem is to examine all possible Hamilton circuits and select one of minimum total length. 
How many circuits do wc have to examine to solve the problem if there are n vertices in 
the graph? Once a starting point is chosen, there are (n - 1 !) different Hamilton circuits 
to examine, since there are n - 1 choices for the second vertex, n - 2 choices for the 
third vertex, and so on. Since a Hamilton circuit can be traveled in reverse order, we 
need only examine (n - 1)!/2 circuits to find our answer. Note that in - i)!/2 grows 


Route 

— 

Total Distance (mites) 

Refroi t -Toledo-Grand Rapids-Sagmaw-Kalamazoo-Detroit 

610 

Detroi t Toledo -Grand Rapids-Kalamazoo-Saginaw-Detroit 

516 

Detro i 1- - To 1 edo- Kalanwoo-S agi ri a w -G ran d Rapj ds- De troi t 

588 

De t r i.i i t-To 1 edo—Ka 1 arnazoo—Grand Rapids-Saginaw-Detroi t 

458 

De trot t-To ledo—S agin aw-Kalama zoo-Grand Rapid s-Dctro it 

540 

De t ro i t-Tol edo-Sagi n av> -Grand Rap j ds- K al amazoo- Detroi t 

504 

Dk Iroit-Sagma w-To ledo-Grand Rap ids- Kal amazoo-Detroi t 

598 

Detroit-Saginaw-Toledo-Kalainazoo-Grand Rapids-Detroit 

576 

Detro it-Sagi na w-Kal amazoo-Tn ledo-Grand R a pids-Detro i t 

682 

Detroi l-S agi n aw-Gr and Rapids-Toledo-Kalamazoo-Detroit 

646 

De t roi t-Grand Rapids-Saginaw-To Iedo-Kalam azoo-De troi t 

670 

De iro i t-Grand R api ds-To ledo-S agi n a w-Kal amazoo-De troi t 

728 
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extremely rapidly. Trying to solve a traveling salesman problem in this way when there 
are only a few dozen vertices is impractical. For example, with 25 vertices, a total of 
241/2 (approximately 3.1 X 10 23 ) different Hamilton circuits would have to be consid¬ 
ered. If it took just 1 nanosecond (10 -9 second) to examine each Hamilton circuit, a 
total of approximately 10 million years would be required to find a minimum-length 
Hamilton circuit in this graph. 

Since the traveling salesman problem has both practical and theoretical importance, 
a great deal of effort has been devoted to devising efficient algorithms that solve it. 
However, no algorithm with polynomial worst-case time complexity is known for 
solving this problem. Furthermore, if a polynomial worst-case time complexity algo¬ 
rithm were discovered for the traveling salesman problem, many other difficult 
problems would also be solvable using polynomial worst-case time complexity 
algorithms {such as determining whether a proposition in n variables is a tautol¬ 
ogy, discussed in Chapter 1). This follows from the theory of NP-eompleteness. 
(For more information about this, consult the references on this topic at the end of 
the text.) 

A practical approach to the traveling salesman problem when there arc many ver¬ 
tices to visit is to use art approximation algorithm. These are algorithms that do not 
necessarily produce the exact solution to the problem but instead are guaranteed to 
produce a solution which is close to an exact solution. That is, they may produce a 
Hamilton circuit with total weight W' so that W ‘ If" ' cWj where W is the total 
length of an exact solution and c is a constant. For example, there is an algorithm with 
polynomial worst-case time complexity such that c = 3/2. In practice, algorithms have 
been developed that can solve traveling salesman problems with as many as 1000 ver¬ 
tices within 2% of an exact solution using only a few minutes of computer time. For 
more information about the traveling salesman problem, including history, applications, 
and algorithms, see the chapter on this topic in Applications of Discrete Mathematics 
fMiRo91 J. 


Exercises 


1. For each ufthe following problems about a subway sys¬ 
tem, describe a weighted graph model that can be used 
to solve the problem. 

a) What is the least amount of time required to travel 
between two stops'? 

b) What is the minimum distance that can be traveled 
to reach a stop from another stop"? 

c) What is the least fare required to travel between two 
stops if fares between stops are added to give the 
total fare 7 

(n Exercises 2-4 find the length of the shortest path between 
a and z in the given weighted graph. 


2. i> 5 d 
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5. What is the shortest path between a and c in each of the 
weighted graphs in Exercises 2—4? 

6. Find the length of the shortest path between the fol¬ 
io wine pairs of vertices in the weighted graph in Exer¬ 
cise 3. 

a) (tandtf b) u and/ c) c and/ di hmdz 

7. Find the shortest paths in the weighted graph in Exer¬ 
cise 3 between the pairs of vertices in Exercise 6. 

8. Find the shortest path (in mileage) between each of the 
following pairs of cities in the airline system show'll in 
Figure 1. 

a) New York and I,os Angelas 

b) Boston and San Francisco 
cl Miami and Denver 

<|i Miami and Los Angeles 

9. Find the combination of flights with the least total air 
lime between the pairs of cities in Exercise 8, using the 
flight times shown in Figure L 

10. Find the least expensive combination of flights con¬ 
necting the pairs of cities in Exercise 8, using the fares 
shown in Figure 1. 

11. Find the shortest route (in distance) between computer 
centers in each of the following pairs of cities in the 
communications network shown in Figure 2. 

a) Boston and Lets Angeles 
h) New York and San Francisco 
cl Dallas and San Francisco 
d) Denver and New York 

12. Find the route w ith the shortest response time between 
the pairs of computer centers in Exercise 11 using the 
response times given in Figure 2. 

13. Find the least expensive route, in monthly lease 
charges, between the pairs of computer centers in Ex¬ 
ercise 11 using the lease charges given in Figure 2, 

14. Explain how to find the path with the least numher of 
edges between two vertices in an undirected graph by 
considering it as a shortest path problem in a weighted 
graph 

15. Extend Dijkstra's algorithm for finding the length of 
the shortest path between two vertices in a weighted 
simple connected graph so that the length of the short¬ 
est path between the vertex a and every other vertex of 
(lie graph is found. 


16. Extend Dijkstra’s algorithm for finding the length of 
the shortest path between two vertices in a weighted 
simple connected graph so that the shortest path be¬ 
tween these vertices is constructed. 

!7. lire weighted graphs in the figures below show some 
major roads in New Jersey. Part fa) shows the distances 
between cities on these roads; part (b) shows the tolls. 



a) Find the shortest route in distance between Newark 
and Camden, and between Newark and Cape May. 
using these roads. 


PDF created with pdfFactory trial version www.pdffactorv.com 











500 7 ! Graphs 


J Mt.nftllHM 1 Flovd’s Algorithm, 

I procedure FloydlG: weighted simple graph) 

i f G has vertices iq, i- 2 .and weights w(v*. v/> with w{v it vj) -- * if (vy, vj) is 

I not a it edge} j 

i for j : - 1 to it ! 

j for j : - 1 to n 
i v ; ): - Mv*. vj) 

for i : ■- 1 to n 
j for j : -~ l to n 
j for k : - l to n 

. if d{Vj, Vj) * d(vj. vjj) < d(vj, v*) then d(vj, t>): =■ 

I d(Vj, t’r) + d{Vi, I'jj) 

| {d(Vi. Vj) is the length of the shortest path between 17 and v y } 


b) Find she least expensive route in terms of total tolls 
using the roads in the graph between the pairs of 
cities in part la') of this exercise, 

18, Is the shortest path between two vertices 111 a weighted 
graph unique if the weights of edges are distinct? 

Id, What are some applications where it is necessary to 
find the length of the longest simple path between two 
veil ices in a weighted graph? 

2(1. What ts the length of the longest simple path in the 
weighted graph in Figure 4 between a and *'■ Between 
r and 

Floyd’s algorithm can be used to find the length of the 
shortest path between all pairs of vertices in a weighted 
connected simple graph. However, this algorithm cannot be 
used to construct shortest paths. (In the following, assign an 
infinite weight to any pair of vertices not connected by an 
edge in the graph.) 

21. Use Floyd's algorithm to find the distance between all 
pairs of vertices in the weighted graph in Figure 4. 

*22. Prove that Floyd’s algorithm determines the shortest 
distance between nil pairs of vertices in a weighted 
simple graph. 

*23. Give a big-(? estimate of the number of operations 
(comparisons and additions) used by Floyd's algo¬ 
rithm to determine the shortest distance between 
every pair of vertices in a weighted simple graph wish 
n vertices. 

*24, Show that Dijkstra’s algorithm may not work if edges 
can have negative weights. 


25. Solve the traveling salesman problem for the follow¬ 
ing graph by finding the total weight of all Hamilton 
circuits and determining a circuit with minimum total 
weight. 


3 h 



26, Solve the traveling salesman problem for the follow¬ 
ing graph by finding the total weight of all Hamilton 
circuits and determining a circuit with minimum total 
weight 


a 3 b 



27, Find the route with the least total airfare that visits each 
of the cities in the following graph g where the weight 
on an edge is the least price available for a flight be¬ 
tween the two cities. 
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28, Find the route with the least total airfare that visits each 
ol the cities m the following graph g where the weights 



on an edge is the least price available for a flight be¬ 
tween the two cities. 

29. Construct a. weighted undirected graph such that the to¬ 
tal weight of a circuit which visits every vertex at least 
once is minimized for a circuit which visits some ver¬ 
tices more than once. (Hint: There are examples with 
three vertices.) 

30. Show that the problem of finding the circuit of min¬ 
imum total weight which visits even' vertex of a 


weighted graph at least once can. be reduced to the 
problem of finding the circuit of minimum total weight 
which visits each vertex of a weighted graph exactly 
once. Do so by constructing a new weighted graph with 
the same vertices and edges as the original graph but 
whose weight of the edge connecting the vertices u and 
i is equal to the minimum total weight of a path from 
u to v in the original graph. 


7.7 _ 

Planar Graphs 


INTRODUCTION 

Consider the problem of joining three houses to each of three separate utilities, as shown 
Tty/) in Figure 1, is it possible to join these houses and utilities so that none of the connec¬ 
tions cross? This problem can be modeled using the complete bipartite graph A' } j, The 
original question can be rephrased as: Can } be drawn in the plane so that no two of 
its edges cross' 7 



FIGURE 1 Three Houses and Three Utilities. 
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KWMPI K 1 



KICl KK 2 Tilt Graph K f . FlCL'KK 3 Ka Drawn with No Crossings. 


In this section we will Study the question of whether a graph can be drawn in the 
plane without edges. In particular, we will answer the houses-and-utilities problem. 

There are always many ways to represent a graph. When is it possible to find at 
least one way to represent this graph in a plane without any edges crossing? 


DEFINITION 1. A graph is called planar if it can be drawn is theplme wt&ettf. 
any edges crossing {where a crossing of edges is the intersection of tbelffleff&Pacs 
representing them at a point other that! their common endpoint). Such a drawing is 
called a planar representation of the graph, 


A graph may be planar even if it is usually drawn with crossings, since it may be 
possible to draw it in a different way without crossings. 


Is Kl i (shown in Figure 2 with two edges crossing) planar ’ 7 

Sahelian: K.\ is planar because it can be drawn without crossings, as shown in Figure 3. 


Is Q 5 , shown in Figure 4, planar? 

Solution: Qi is planar, because it can be drawn without any edges crossing, as shown 
in Figure 5. ■ 



KICKRK4 The Graph £> v 
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EXAMPLE 3 



We can show that a graph is planar by displaying a planar representation. It is harder 
to show that a graph is nonplanar. We will give an example to show how this can be 
done in an ad hoc fashion. Later we will develop some general results that can be used 
to do this. 


Is K. shown in Figure 6 . planar.’ 

Solution: Any attempt to draw AV, in the plane with no edges crossing is doomed. 
We now show why. In any planar representation of A^j. Lhc vertices vy and v -2 must 
he connected to both Va and i\, These four edges form a closed curve that splits the 
plane into two regions, R] and AS, as shown in Figure 7(a). The vertex is in either 
R l or A;. When vj is in /?;, the inside of the closed curve, the edges between v 3 and 
and between v.i and V 5 separate A; into two subregions, R->] and as shown in Fig¬ 
ure 7(b). 

Next, note that there is no way to place the final vertex v f , without forcing a crossing. 
For if v 6 is in R } , then the edge between % and vj cannot be drawn without a crossing. 
If v ( , is in AS], then the edge between V 2 and vg cannot be drawn without a crossing. If 
p, is in R 2 2 , then the edge between vp and v& cannot he drawn without a crossing. 

A similar argument can be used when is in R ]. The completion of this argument 
is left for the reader (see Hxereise 8 at the end of this section). It follows that Ah > is not 
planar, ■ 


Example 3 solves the utilities-and-houses problem that was described at the beginning 
of this section. The three houses and three utilities cannot be connected in the plane 
without a crossing A similar argument can be used to show that AT is nonplanar. (See 
Exercise 9 at (he end of this section.) 


ELLER‘S FORMULA 

A planar representation of a graph splits the plane into regions, including an unbounded 
region. For instance, the planar representation of the graph shown in Figure 8 splits the 
plane into six regions. These are labeled in the figure, Euler showed that all planar rep¬ 
resentations of a graph split the plane into the same number of regions. He accomplished 
this by finding a relationship among the number of regions, the number of vertices, and 
the number of edges of a planar graph. 
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FIGURE 8 The Regions of (he Planar Representation of a Graph. 


THEOREM 1 EULER’S FORMULA Let G be a connected planar simple graph with e edges 

and v vertices. Let r be the number of regions in a planar representation of G. Then 

r - e - v + 2. 


K, 


FIGURE 4 The 
Basis Case of the 
Proof of Euler’s 
Formula. 


Proof: First, we specify a planar representation of G. We will prove the theorem by 
eonsnuctinj; a sequence of subgraphs G\. Gy, — G„ = G, successively adding an edge 
at each stage. This is done using the following uwskiim 1 definition. Arbitrarily pick 
one edge of G to obtain Gi. Obtain G„ from G„. i by arbitrarily aiiumg ar edge, rna: i' 
incident with a vertex already in G„. t , adding the other vertex incident with this edge it 
it is not already in G„- [. This construction is possible since G is connected. G is obtained 
after e edges are added. Let r„, e„, and v« represent the number of regions, edges, and 
vertices of the planar representation of G„ induced by the planar representation of G. 
respectively. 

The proof will now proceed by induction, The relationship rq - e\ - iq - 2 is true 
for G], since «i = I. iq =2, and n = I. This is shown in Figure 9. 

Now assume that r„ = e„ - v„ + 2, Let {a n ~ i, b R+ (} be the edge that is added to G n 
to obtain G n +\ . There are two possibilities to consider. In the first case, both a„ , ■ and 
b„ + ] are already in G„- These two vertices must be on the boundary of a common region 
R, or else it would be impossible to add the edge {fl n+] , i>„, i} to G„ without two edges 
crossing ( and G„ 1 1 is planar). The addition of this new edge splits R into two regions. 
Consequently, in this case, r ttT i = r n + l, e n+J ~ e n + J, and v n+] - r„. Thus, each 
side of the formula relating the number of regions, edges, and vertices increases by 
exactly one, so this formula is still true. In other words, r B+1 - e fl+] - v w _j + 2. This 
ease is illustrated in Figure 10(a). 

In the second case, one of the two vertices of the new edge is not already in G n , 
Suppose that a n ^\ is in G„ but that b„ j is not. Adding this new edge does not produce 



_on_fb) 

FIGURE It) Adding an Edge to G„ to Produce (?„. j. 
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any new regions, since £>„h must be in a region that has a „| on its boundary. Con¬ 
sequently, r„.) — r„. Moreover, e„ +! - r,+ 1 and = v„ + 1. Each side of the 
formula relating the number of regions, edges, and vertices remains the same, so the 
formula is still true. In other words, | = e„+] - vv] T 2. This case is illustrated in 
Figure 10(b). 

We have completed the induction argument. Hence r„ = e n - v„ -f 2 for all ft. Since 
the original graph is the graph G,.. obtained after e edges have been added, the theorem 
is true. □ 

Euler's formula is illustrated in the following example. 


EXAMP1.E 4 Suppose that a connected planar simple graph has 20 vertices, each of degree 3. Into 

how many regions does a representation of this planar graph split the plane? 

Solution: This graph has 20 vertices, each of degree 3, so that v - 20. Since the sum 
of the degrees of the vertices, 3v - 3 - 20 - 60, is equal to twice the number of edges. 
It, we have 2e - 60, or e — 30. Consequently, from Euler's formula, the number of 
regions is 

r — f - v t 2 = 30 - 20 + 2 - ! 2 . ■ 


Euler's formula can be used t» establish some inequalities that must be satisfied by 
planar graphs. One such inequality is given in the following corollary. 

COROl 1 ARY 1 If G is a connected planar simple graph with e edges and v vertices where v — 3, 
then e ri 3v - 6. 

The proof of Corollary I is based on the concept of the degree of a region, which 
is defined to be the number of edges on the boundary' of this region. When an edge 
occurs tw ice on the boundary (so that it is traced out twice when the boundary is traced 
out), it contributes 2 to the degree. The degrees of the regions of the graph shown in 
Figure 11 are displayed in the figure. 

The proof of Corollary 1 can now be given. 


f: 
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EXAMPLE 5 


COROLLARY 2 


EXA MPI.E a 


Proof: A connected planar simple graph drawn in the plane divides the plane into re¬ 
gions, say rof them. The degree of each region is at least 3, (Since the graphs discussed 
here are simple graphs, no multiple edges that could produce regions of degree 2, or 
loops that could produce regions of degree 1, arc permitted.) In particular, note that the 
degree of the unbounded region is at least 3 since there are at least three vertices in the 
graph. 

Note that the sum of the degrees of the regions is exactly twice the number of 
edges in the graph, because each edge occurs on the boundary of a region exactly twice 
(either in two different regions, or twice in the same region). Since each region has 
degree greater than or equal to 3, it follows that 

2c - V deg(R) - 3r. 
all regions R 

Hence, 

ame r. 

Using r - e — v -r 2 (Euler’s formula), we obtain 
e - v + 2 (2/3 ; . e. 

It follows that e/3 s v - 2. This shows that e < 3v - 6. □ 

This corollary can be used to demonstrate thin Ky is nonplanar. 


Show that X) is nonplanar using Corollary 1. 

Solution: The graph X' 5 has five vertices and 10 edges. However, the inequality e ^ 
3i J - 6 is not satisfied for this graph since e = 10 and 3v -6 = 9. Therefore. K 5 is not 
planar | 


It was previously shown that Kj.j is not planar. Note, however, that this graph has 
six vertices and nine edges. This means that the inequality e - 9 < 12-3*6-6 
is sanstied. Consequently, the fact that the inequality e ^ 3v - 6 is satisfied does nor 
imply that a graph is planar. However, the following corollary of Theorem I can be used 
to show that Ky? is nonplanar. 

If a connected planar simple gr aph has e edges and v vertices with v > 3 and no 
circuits of length 3, then e s 2v - 4. 

The proof of Corollary 2 is similar to that of Corollary 1, except that in this case the 
fact that there are no circuits of length 3 implies that the degree of a region must be at 
least 4, The details of this proof are left for the reader (see Exercise i 3 at (he end of this 
section,), 


Use Corollary 2 to show that Kfj is nonplanar. 

Solution: Since K u has no circuits of length 3 (this is easy to see since it is bipartite). 
Corollary 2 can he used. Ky ■, has six vertices and nine edges. Since e - 9 and 2v - 4 «= 
8, Corollary 2 shows that AVi is nonplanar. g 
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FIG! RF, 12 Homcomorphie Graphs, 



KURATOWSKI’S THEOREM 

We have seen that j and A'g are not planar. Clearly, a graph is not planar if it contains 
either of these two graphs as a subgraph. Furthermore, all nonplanar graphs must con¬ 
tain a subgraph that can be obtained from Wjj or K< using certain permitted operations. 

If a graph is planar, so will be any graph obtained by removing an edge {it, r } and 
adding a new vertex w together with edges {u, w} and {w, v}, Such an operation is called 
an elementary subdivision. The graphs G] ~ (V'j, and G 2 ~ (V 2 , F - : ) are called 
homeomorphic if they can be obtained from the same graph by a sequence of el¬ 
ementary subdivisions. The three graphs displayed in Figure 12 arc homeomorphic, 
since all can be obtained from the first graph by elementary subdivisions. (The reader 
should determine the sequences of elementary subdivisions needed to obtain Gj and Gj 
from G|.) 

The Polish mathematician Kuratowski established the following theorem in 1930, 
which characterizes planar graphs using the concept of graph humemnorphism, 

THEOREM 2 A graph is nonplanar if and only if it contains a subgraph homeomorphic to AT 3 or 

Ky 

It is clear that a graph containing a subgraph homeomorphic to K 33 or AT is nonpla- 
nar. However, the proof of the converse, namely that every nonplanar graph contains a 
subgraph homeomorphic to K-,y or AT. is complicated and will not be given here. The 
following examples illustrate how Kuratowski’s theorem is used. 


HXAM1M.K 7 Determine whether the graph G shown in Figure 13 is planar. 

Solution: G has a subgraph H homeomorphic to A’ s H is obtained by deleting h, /, and 
k and all edges incident with these vertices. H is homeomorphic to AT since it can be 


Kazfmlerz Kuratowski <1896-1980k Kazimier/ Kuratowski. the son of a famous Warsaw lawyer, at¬ 
tended secondary school in Warsaw. He studied in Glasgow, Scotland, from 1913 to 1914 but emild not 
return there after the outbreak of World War I. In 1915 he entered Warsaw University, where be was active 
in the Polish patriotic student movement. He published his first paper in 1919 and received his Ph.D. in 
1921. He was an active member of the group known as the Warsaw School of Mathematics, working in the 
areas of the foundations of set theory and topology. He was appointed associate professor at the Lwbw Poly- 
technical University, where he stayed for 1 years, collaborating with the important Polish mathematicians 
Banach and Warn, in 1930, while at Lwdw, Kuratowski completed his work characterizing planar graphs. 

In 1934 he returned to Warsaw University as a full professor. Until the start of World War II. he was 
active in research and teaching. During the war. because of the persecution of educated Poles, Kuratowski 
went into hiding under an assumed name and taught at the clandestine Warsaw University. After the war 
he helped revive Polish mathematics, serving as director of the Polish National Mathematics Institute. He 
wrote over ISO papers and three widely used textbooks. 
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a b 



a b 



A 5 . 


FIGURE 13 The Undirected Graph G, a Subgraph H Homeomorphir to K$, and K-,. 



obtained from K$ (with vertices a, b, r. g, and t) by a sequence of elementary subdivt- 
sions, adding the vertices d, e, and/'. (The reader should construct such a sequence, of 
elementary subdivisions.) Hence, G is ncmplanar. ■ 


EXAMPLE 8 Is the Petersen graph, shown in Figure 14(a), planar? (The Danish mathematician Julius 

Petersen introduced this graph in 1891; it is often used to illustrate various theoretical 
properties of graphs.) 

Solution: The subgraph H of the Petersen graph obtained by deleting b and the three 
edges that have b as an endpoint, shown in Figure 14(b), is homeomorphic lo ATy, with 
vertex sets {/, d, j} and {t\ i, h}, since it can be obtained by a sequence of elementary 
subdivisions, deleting {d, h } and adding (c, A} and {c. d}, deleting {e, /} and adding {a, e} 
and {a, /}, and deleting {i, j} and adding [g, j} and {g, j}, Hence, the Petersen graph is 
not planar. ■ 


Exercises 

J- Can five houses be connected to two utilities without In Exercises 2-4 draw the given planar graph without any 
connections crossing? crossings. 
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In Exercises 5-7 determine whether the given graph is pla¬ 
nar. If so, draw it so that no edges cross. 



8 . Complete the argument in Example 3 
ft. Show that K 5 is nonplanar using an argument similar 
to that given in Example 3. 

10. Suppose that a connected planar graph has eight ver¬ 
tices, each of degree 3. Into how many regions is the 
plane divided by a planar representation of this graph? 

11. Suppose that a connected planar graph has six vertices, 
each of degree 4. Into how many regions is the plane 
divided by a planar representation of this graph? 


12. Suppose that a connected planar graph has 30 edges. If 
a planar representation of this graph divides the plane 
into 20 regions, how many vertices does this graph 
have? 

13. Prove Corollary 2. 

14. Suppose that a connected bipartite planar simple graph 
has e edges and v vertices. Show 1 that e s 2v - 4 if 

V Si: 3 . 

*15. Suppose that a connected planar simple graph with 
e edges and v vertices contains no simple circuits of 
length 4 or less. Show that e £ (5/3)v - (10/31 if 
r - 4. 

16. Suppose that a planar graph has k connected compo¬ 
nents, f edges, and v vertices. Also suppose that the 
plane is divided into r regions by a planar representa¬ 
tion of the graph. Find a formula for t in terms of e. v, 
and k. 

17, Which of the following nonplanar graphs have the 
property that the removal of any vertex and all edges 
incident with that vertex produces a planar graph? 

a) Sx hi Kf, c) K u d) K Xi 

[n Exercises 18-20 determine whether the given graph is 

houteoinorphic to Jfy j, 


IS. 
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In Exercises. 21 -23 use Kuratowski's theorem to determine 
whether the given graph is planar 

21 . U h c ii 



The crossing number of a simple graph is the minimum 
number of crossings that can occur when this graph is drawn 
in the plane where no three arcs representing edges are per¬ 
mitted to cross at the same point. 


24. Show that Aj„i has I as its crossing number. 

**25. Find the crossing numbers of each of the followi rig non- 
planar graphs. 

a) A s b) K(, e) K- 

d.i e) A 4 4 f) A's,s 

*26. Find the crossing number of the Petersen graph. 

**27. Show that if m and n are even positive integers, 
the crossing number of K m- „ is less than or equal to 
mnim - 2)1 n - 2)/16. {Hint: Place m vertices along the 
x-axis so that they are equally spaced and symmetric 
about the origin and place n vertices along the y-axis 
so that they are equally spaced and symmetric about 
the origin, Now connect each of the m vertices on the 
x-axis to each of the vertices on the y-axis and count 
the crossings.) 

The thickness of a simple graph G is the smallest number 

of planar subgraphs of G that have G as their union. 

28. Show that AT 3 has 2 as its thickness. 

*29. Find the thickness of the graphs in Exercise 25, 

30. Show that if G is a connected simple graph with v ver¬ 
tices and e edges, then the thickness of G is at least 
M3v - fill. 

*31. Lise Exercise 30 to show that the thickness of K„ is tit 
least i(n + 7)/6, whenever n is a positive integer. 

32. Show that if (7 is a connected simple graph with v ver¬ 
tices and r edges and no circuits of length three, then 
the thickness of 6 ' is at least [t?/( 2 v - 4jl. 

33. I. Jse Exercise 32 to show that the thickness of K m „ is at 
least \mnH2m + 2 /; 4)1 whenever m and n are posiiti vc 
integers. 

*34, Draw AT on the surface of a torus (a doughnut-shaped 
solid) so no edges cross. 

*35. Draw An on the surface of a torus SO no edges cross. 


18 

Graph Coloring 


INTRODUCTION 


Problems related to the coloring of maps of regions, such as maps of parts of the world, 
M'eb lutvc generated many results in graph theory. When a map* is colored, two regions 
with a common border are customarily assigned different colors. One way to ensure 


will Hsu umc that all regions in a niap arc connected. This e linn n ate >; any problem?* presented by such 
geographical entilies as Michigan. 
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FIGURE I Two Maps. 


that two adjacent regions never have the same color is to use a different color fur each 
region. However, this is inefficient, and on maps with many regions it would be hard to 
distinguish similar colors. Instead, a small number of colors should be used whenever 
possible. Consider the problem of determining the least number of colors that can be 
used to color a map so that adjacent regions never have the same color. For instance, 
for the map shown on the left in Figure 1, four colors suffice, but three colors are not 
enough. (The reader should check this.) In the map on the right in Figure 1, three colors 
are sufficient (but two are not). 

Each map in the plane can be represented by a graph. To set up this correspondence, 
each region of the map is represented by a vertex. Edges connect two vertices if the 
regions represented by these vertices have a common border. Two regions that touch 
at only one point are not considered adjacent. The resulting graph is called the dual 
graph of the map. By the way in which dual graphs of maps are constructed, it is clear 
that any map in the plane has a planar dual graph. Figure 2 displays the dual graphs 
that correspond to the maps shown in Figure i. 

The problem of coloring the regions of a map is equivalent to the problem of col¬ 
oring the vertices of the dual graph so that no two adjacent vertices in this graph have 
the same color. We give the following definition. 

DEFINITION 1 , A coloring of a simple graph is the assignment of a color to each 
vertex of the graph so that no two adjacent vertices are assigned the same color. 

A graph can be colored by assigning a different color to each uf ils vertices. How¬ 
ever, for most graphs a coloring can be found that uses fewer colors than the number of 
vertices in the graph. What is the least number of colors necessary? 


B 



FIGURE 2 Dual Graphs of the Maps in Figure 1. 
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BfiflNITfON 2. The chromatic mntber of a graph is tfi? least number of cotore 
needed fora coioriag ef this graph. 


Note that asking for the chromatic number of a planar graph is the same as asking 
for the minimum number of colors required to color a planar map so that no two adjacent 
regions are assigned the same color. This question has been studied for more than 100 
years. The answer is provided by one of the most famous theorems in mathematics. 

THEOREM 1 The Four Color Theorem The chromatic number of a planar graph is no greater 

than four. 


The four color theorem was originally posed as a conjecture in the 1850s. Il was 
finally proved by the American mathematicians Kenneth Appel and Wolfgang Haken 
in 1976. Prior to 1976, many incorrect proofs were published, often with hard-to-find 
errors. In addition, many futile attempts were made to construct counterexamples by- 
drawing maps that require more than four colors. 

Perhaps the most notorious fallacious proof in all of mathematics is an incorrect 
proof of the four color theorem published in 1879 by a Loudon hart:sic: and ama¬ 
teur mathematician, Alfred Kempe. Mathematicians accepted his proof as correct until 
1890, when Percy Heawood found an error that made Kempe *5 argument incomplete. 
However, Kempe’s line of reasoning turned out to be the basis of the successful proof 
given by Appel and Haken Their proof relies on a careful case-by-case analysis carried 
out by computer. They showed that if the four color theorem were false, there would 
have to be a counterexample of one of approximately 2000 different types, and they 
then showed that none of these types could lead to a counterexample. They used over 
1000 hours of computer time in their proof. This proof generated a large amount of 
controversy, since computers played such an important role in it. For example, could 
there be an error in a computer program that led to incorrect results? Was their argument 
really a proof if it depended on what could be unreliable computer output? 

The four color theorem applies only to planar graphs. Nonplanar graphs can have 
arbitrarily large chromatic numbers, as will be shown in Example 2. 

Two things are required to show that the chromatic number of a graph is n. First, we 
must show that the graph can be colored with n colors. This can be done by constructing 


Alfred Bray Kempe (1849-1921). Kempe was a banister and a leading authority on ecclesiastical law. 
. However, hav mg studied mathematics at Cambridge University, be retained his interest in it, and later in life 
he devoted considerable time to mathematical research. Kempe made contributions to kinematics, the branch 
of mathematics dealing with motion, and tn mathematical logic However. Kempe is best remembered for 
his Fallacious proof of the four color theorem 

Historical Note: Ln 1852, an ex-student of Augustus Dc Morgan, Francis Guthrie, noticed that the coun¬ 
ties in England could be colored using four colors so that no adjacent counties were assigned the same color. 
On this evidence, he conjectured that the tour color theorem was true. Francis told his brother Frederick, at 
that time a student of De Morgan, about this problem. Frederick in turn asked his teacher De Morgan about 
his brother's conjecture, De Morgan was extremely interested in this problem and publicized it throughout 
the mathematical community. In fact, the first written reference to the conjecture can be found in a letter from 
De Morgan to Sir William Rowan Hamilton. Although De Morgan thought Hamilton would be interested 
in this problem, Hamilton apparently was not interested in it, since it had nothing to do with quaternions. 
Historical Note: Although a simpler proof of the four color theorem was found hy Robertson, Sanders. 
■ ifedK Seymour, and Thomas in 1996, reducing the computational part of the proof to examining 635 configure* 

■ V tions, no proof that docs not rely on extensive computation has yet been found. 
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EXAMPLE 1 




H 


FIGURE 3 The Simple Graphs G and H. 


such a coloring. Second, we must show that the graph cannot be colored using fewer 
than n colors. The following examples illustrate how chromatic numbers can be found. 


What are the chromatic numbers of the graphs G and M shown in Figure 3? 

Solution: The chromatic number of G is at least 3, since the vertices a, b, and t: must 
be assigned different colors. To see if G can be colored with three colors, assign red to 
a , blue to b, and green to c. Then, d can (and must) be colored red since it is adjacent 
to b and c. Furthermore, e can (and must) he colored green since it is adjacent only to 
vertices colored red and blue, and / can (and must) be colored blue since it is adjacent 
only to vertices colored red and green. Finally, g can (and must) be colored red since it 
is adjacent only to vertices colored blue and green. This produces a coloring of G using 
exactly three colors. Figure 4 displays such a coloring. 

The graph H is made up of the graph G with an edge connecting a and g. Any 
attempt to color H using three colors must follow the same reasoning as that used to 
color G, except at the last stage, when all vertices other than g have been colored. Then, 
since g is adjacent (in H) to vertices colored red, blue, and green, a fourth color, say 
brown, needs to be used. Hence, H has a chromatic number equal to 4. A coloring of // 
is shown in Figure 4. ■ 




C H 

FIGURE 4 Colorings of the Graphs G and H. 
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EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


a Red b Blue 



FIGURE 5 A Coloring of A'j. 


a Red b Red (Red 



i Blue Blue /Blue ,K Blue 

FIGURE 6 A Coloring of A 3 4 * 


What is the chromatic number of K„ 1 

Solution: A coloring of K n can be constructed using n colors by assigning a different 
coin] to each vertex. Is there a coloring using fewer colors'? The answer is no. No two 
vertices can be assigned the same color, since every two vertices of this graph are ad¬ 
jacent. Hence, the chromatic number of K„ = n. (Recall that K„ is not planar when 
n ■ 5, so this result does not contradict the four color theorem.) A coloring of K~ using 
five colors is shown in Figure 5. ■ 


What is the chromatic number of the complete bipartite graph K,„„, where m and n arc 
positive integers'? 

Solution: The number of colors needed may seem to depend on m and 11 . However, 
only two colors are needed. Color the set of m vertices with one color and the set of 
n vertices with a second color. Since edges connect only a vertex from the set of m 
vertices and a vertex from the set of u vertices, no two adjacent vertices have the same 
color. A coloring of Afi 4 with two colors is displayed in Figure 6 . 1 

Every connected bipartite simple graph has a chromatic number of 2, or 1. since 
the reasoning used in Example 3 applies to any such graph. Conversely, every graph 
with a chromatic number of 2 is bipartite. (See Exercises 23 and 24 at the end of this 
section.) 


What is the chromatic number of the graph C,,? (Recall that C n is the cycle with n 
vertices.) 

Solution: We will first consider some individual cases. To begin, let n = 6 . Pick a 
vertex and color it red. Proceed clockwise in the planar depiction of Q shown in Figure 
7. It is necessary to assign a second color, say blue, to the next vertex reached. Continue 
in the clockwise direction; the third vertex can be colored red, the fourth vertex blue, 
and the fifth vertex red. Finally, the sixth vertex, which is adjacent to the first, can be 
colored blue. Hence, the chromatic number of ff is 2. Figure 7 displays the coloring 
constructed here. 

Next, let n = 5 and consider (. 5 . Pick a vertex and color it red. Proceeding clock¬ 
wise, it is necessary to assign a second color, say blue, to the next vertex reached, 
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HCit'RK 7 Colorings of C; and C 6 - 


Continuing in the clockwise direction, the third vertex can be colored red, and the fourth 
vertex can be colored blue, The fifth vertex cannot be colored either red or blue, since 
it is adjacent to the fourth vertex and the first vertex. Consequently, a third color is 
required for this vertex. Note that we would have also needed three colors if we had 
colored vertices in the counterclockwise direction. Thus, the chromatic number of C-, 
is 3. A coloring of Cj using three colors is displayed in Figure 7. 

In general, two colors arc needed to color C n when n is even. To construct such 
a coloring, simply pick a vertex and color it red. Then proceed around the graph in 
a clockwise direction fusing a planar representation of the graph) coloring the second 
vertex blue, the third vertex red, and so on. The «th vertex can be colored blue, since the 
two vertices adjacent to it. namely the (rt - 1 )st and the fi rst vertices, are both colored red 

When n is odd anti it > 1, the chromatic number of C„ is 3. To see this, pick an in¬ 
itial vertex. To use only two colors, it is necessary to alternate colors as the graph is tra- 
versed in aclockwi.se direction. However, the «th vertex reached is adjacent to two vertices 
of different colors, namely, the first and (n 1 )st. Hence, athird color must be used. E 


The best algorithms known for finding the chromatic number of a graph have ex- 
H'l’h ponential worst-case time complexity (in the number of vertices of the graph). Even the 
problem of finding an approximation to the chromatic number of a graph is difficult. It 
has been shown that if there were an algorithm with polynomial worst-ease time com¬ 
plexity which could approximate the chromatic number of a graph up to a factor of 2 
(that is, construct a bound which was no more than double the chromatic number of the 
graph), then an algorithm with polynomial worst-case time complexity for finding the 
chromatic number of the graph would also exist. 


APPLICATIONS OF GRAPH COLORINGS 


Graph coloring has a variety of applications to problems involving scheduling and as¬ 
signments. Examples of such applications will be given here. The first application deals 
with the scheduling of final exams. 


KXA.MPLlr 5 Scheduling Final Exams How can the final exams at a university be scheduled so that 

tip student has two exams at the same time? 

Solution: This scheduling problem can be solved using a graph model, with vertices 
representing courses and with an edge between two vertices if there is a common student 
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i 



FIGURE S The Graph Representing the Scheduling of Final Exams. 


i n the courses they represent. Each time slot for a final exam is represented by a different 
color. A scheduling of the exams corresponds to a coloring of the associated graph. 

For instance, suppose there are seven finals to be scheduled. Suppose the courses 
are numbered I through 7. Suppose that the following pairs of courses have common 
students: 1 and 2,1 and 3,1 and 4, 1 and 7, 2 and 3,2 and 4,2 and 5,2 and 7, 3 and4, 3 
and 6 ,3 and 7,4 and 5,4 and 6 ,5 and 6 .5 and7, and 6 and 7, In Figure 8 the graph asso¬ 
ciated with this set of classes is shown. A scheduling consists of a coloring of this graph. 

Since the chromatic number of this graph is 4 (the reader should verify this), four 
time slots are needed. A coloring of the graph using four colors and the associated 
schedule are shown in Figure 9. ■ 


Now consider an application to the assignment of television channels. 


Frequency Assignments Television channels 2 through 13 are assigned to stations in 
North America so that no two stations within 150 miles can operate on the same channel. 
How can the assignment of channels be modeled by graph coloring? 

Solution: Construct a graph by assigning a vertex to each station. Two vertices are con¬ 
nected by an edge if they are located within 150 miles of each other. An assignment of 
channels corresponds to a coloring of the graph, where each color represents a different 
channel. ■ 


1 Red 



FIGURE 9 Using a Coloring to Schedule Final Exams. 


C'niLDilTh 

Is 


4, 7 


PDF created with pdfFactory trial version www.pdffactorv.com 










7,8 Exercises 517 


An application of graph coloring to compilers follows. 


EXAMPLE 7 Index Registers In efficient compilers the execution of loops is speeded up when fre¬ 

quently used variables are stored temporarily in index registers in the centra! process¬ 
ing unit, instead of in regular memory. For a given loop, how many index registers are 
needed? This problem can be addressed using a graph coloring model. To set up the 
model, let each vertex of a graph represent a variable in the loop. There is an edge be¬ 
tween two vertices if the variables they represent must be stored in index registers at the 
same time during the execution of the loop. Thus, the chromatic number of the graph 
gives the number of index registers needed, since different registers must be assigned 
to variables when the vertices representing these variables are adjacent in the graph,■ 


Exercises 


1. Construct the dual graphs for each of the following 
maps, 



2. Find the number of colors needed to color the maps 
in Bxerci.se 1 so that no two adjacent regions have the 
same color. 

In Exercises 3--V find the chromatic number of the given 
graph 
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10. For iht graphs in Exercises 3-9. decide whether it i.s 
possible to decrease the chromatic number by remov¬ 
ing a single vertex and all edges incident with it. 

U. Wiik'h graphs have a chromatic number of 1? 

12. What is the least number of colors needed to color a 
map of the United States? Do not consider adjacent 
states that meet only at a comer. Suppose that Michi¬ 
gan is one region. Consider the vertices representing 
Alaska and Hawaii as isolated vertices. 

13. What is the chromatic number of VV„? 

14. Show that a simple graph which has a circuit with an 
odd ilumber of vertices in it cannot be colored using 
twocolois, 

15. Schedule the final exams for Math 115, Malh ) 16, Math 
IK5, Math 1 %, CS 10!, CS 102, CS 273, and CS 473. 
using the fewest number of different time slots, if there 
arc no students taking both Math 115 and CS 473, both 
Malh 116 and CS 473, both Math 195 and CS 101, both 
Math 195 and CS 102. both Math 115 and Math 116, 
both Math 115 and Math 185, anti both Math 1X5 and 
Malh 195, but there are students in every other combi¬ 
nation of courses. 


16. How many different channels are needed for stx sta¬ 
tions located at the distances shown in the table, if two 
stations cannot use the same channel when they are 
within 150 miles of each other? 1 



1 

2 

3 

4 

5 

6 

1 

- 

85 

175 

200 

50 

too 

2 

85 


125 

175 

too 

Iff) 

! J 

175 

125 

- 

ioo 

200 

250 

4 

200 ‘ 175 

\ 

too : - 

210 

220 

\ 5 

51) 

too 

2(X) : 210 

.. i 

- 

100 

h 

[t» 

160 

251) 

_ 

720 

l 

too 



17. The mathematics department has six committees that 
meet once a month. How many different meeting, times 
must be used to ensure that no one is scheduled to be 
at two meetings tit the same time if the committees are 
<?j = ; {Arlinghaus. Brand, Zaslavsky], C 2 - {Brand, 
U't:, Kami]. Ci {Arlinghaus, Rosen , Zaslavsky], 
Ci = {Lee, Rosen, Zaslavsky}, Cjj - {Arlinghaus, 
Brand], and Q. ^ { Brand, Rosen, Zaslavsky]. 

18. A zoo wants to set up natural habitats in which to ex¬ 
hibit its animals. Unfortunately, some animals will eat 
some of the others when given the opportunity. How 
can a graph model and a coloring be used to determine 
the number of different habitats needed and the place¬ 
ment of the animals in these habitats'* 

An edge coloring of a graph is an assignment of colors to 
edges so that edges incident with a common vertex arc as¬ 
signed different colors. The edge chromatic number of a 
graph i.s the smallest number of colors that can be used in 
an edge coloring of the graph. 

19. Find the edge chromatic numbers of each of the graphs 
in Exercises 3-9, 

*20. Find the edge chromatic numbers of 
a) K„. b )K„'„, c)C„, d) IV,. 

21. Seven variables occur in a loop of a computer program, 
The variables and the steps during which they must be 
stored are: /: steps 1 through 6; u: step 2; v. steps 2 
through 4: vt-r steps 1,3, and 5; x: steps I and 6; y: steps 
3 through 6; and & steps 4 and 5. How many differ¬ 
ent index registers are needed to store these variables 
during execution? 

22. What can be said about the chromatic number of a 
graph that has K „ as a subgraph? 
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23. Show that ;t simple graph with a chromatic number of 
2 is bipartite. 

24. Show that a connected bipartite graph has a chromatic 
number of 2 

The following algorithm can be used to color a simple 
graph. First, list the vertices iy, vj, vj,.... i'„ in order of 
decreasing degree so rh;i( degfvj) degfoj & ••• o: 
tlegt v„l- Assign color 1 to v ; and to the next vertex in the list 
not adjacent to vy (if one exists), and successively to each 
vertex in the list not adjacent to a vertex already assigned 
color 1. Then assign color 2 to the first vertex in the list 
not already colored. Successively assign color 2 to vertices 
in the list that have not been already colored and arc not 
adjacent to vertices assigned color 2. If uncolored vertices 
remain, assign color 3 to the first vertex in the list not yet 
colored, and use color 3 to successively color those vertices 
not already colored and not adjacent to vertices assigned 
color 3, Continue this process nntil all vertices are colored, 

25. Construct a coloring of the following graph using this 
algorithm. 


:! h 



*26. Use pseudocode to describe this coloring algorithm. 
*27. Show that the coloring produced in this algorithm may 
use mure colors than lire necessary to color a graph. 

A £ -tuple coloring of a graph G is an assignment of a.set of 
k different colors to the vertices of G so that no two adjacent 


vertices are assigned a common color. We denote by ,V.t(G) 
the smallest positive integer n such that <j has a it-tuple 
coloring using ti colors. For example. h'lCA - 4. To sec 
this, note that using only four colors we can assign two col¬ 
ors to each vertex of C t , as illustrated, so that no two adja¬ 
cent vertices are assigned the same color. Furthermore, no 
tower than four colors suffice because the vertices iq and 
tj each must be assigned two colors, and a common color 
cannot be assigned to both i [ and v;. (For more information 
about k -tuple coloring, see [MiRo90].) 

I fed- blue) it is {green, vifewi 

T-f 


igrecn, yellow} y. v ; j red. blue’} 

28. Find the following values: 
a ) t -1A ) 1] i j (Ka ) 

C) Aj(IVa) d)dn(Cs) 

CfU.lA • ! f > A*(a V.I 

”g) V*f i h,if.iA,s) 

*29, Lei G and H be the graphs displayed m Figure 3. Find 
a ) 42(G)- b) v '// i. c) v.itij (1 1 x - H) 

30, What is Xi r G) if G is a bipartite graph and k is a positive 
integer? 

3t. Frequencies for muhile radio (or cellular) telephones 
are assigned by wines. Each /.one is assigned a set of 
Ircqucncics to be used by vehicles in that /one. The 
Same frequency cannot be used in zones where inter¬ 
ference is a problem. Explain bow a Jc-ruple coloring 
can be used to ussign k frequencies to each mobile 
radio zone in a region. 


Key Terms and Results 

TERMS 

undirected edge: ais edge associated to a set r>, where 
k and care vertices 

directed edge: an edge associated to an ordered pair I it, v}. 
where u and v are vertices 

multiple edges: distinct edges connecting the same ver- 

UCtiS 

loop: an edge connecting a vertex with itself 
undirected graph: a sot of vortices together with a sot of 
undirected edges that connect these vertices, 
simple graph: an undirected graph with no multiple edges 
or loops 


multigraph: an undirected graph that may contain multi¬ 
ple edges but no loops 

pseudograph: an undirected graph that may contain mul¬ 
tiple edges and loops 

directed graph: a set of vertices together with a set of 
directed edges that connect these vertices 
directed multigraph: a graph with directed edges that 
may contain multiple directed edges 
adjacent: two vertices are adjacent if there is an edge 
between them 

incident: an edge is incident with a vet lex if the vertex E 
an endpoint of that edge 
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deg(r) (degree of the vertex v in an undirected 
graph): the number of edges Incident with v 
deg"(v) (the in-degree of the vertex v in a graph with di¬ 
rected edges): the number it I edges with v as their ter¬ 
minal vertex 

deg * I. v) (the out-degree of the vertex v in a graph with 
directed edges): the number of edges with r as their ini¬ 
tial vertex 

underlying undirected graph of a graph with directed 

edges: the undirected graph obtained by ignoring the di¬ 
rections of the edges 

A'„ (aimp[etc graph on n vertices): the undirected graph 
With n vertices where each pair of vertices is connected 
by an edge 

bipartite graph; a graph with vertex set partitioned into 
subsets V and V j such that each edge connects a vertex 
in V, and a vertex in If 

Arn.n (complete bipartite graph): die graph with vertex 
set partitioned inlo a subset of m elements and a subset 
of n elements such that two vertices are connected by an 
edge if and only if one is in the first subset and the other 
is in the second subset 

C a (cycle of size n). rt --- 3; the graph with n vertices 

*MV.f s and edges jt»i,v>}. {v^tof.fo_j. >•„}■ 

iv,,r ; ) 

If., (wheel of size n), n v 3: tiw graph obtained from C„ 
by adding a vertex and edges from this vertex to the orig¬ 
inal vertices in C, 

Q r . i/t-cube), n -• t: the graph that has the 2" bit strings of 
length n as its vertices and edges connecting every pair 
of bit strings that differ by exactly one bit 
isolated vertex: a vertex of degree 0 
pendant vertex: a vertex of degree 1 
regular graph: a graph where all vertices have the same 
degree 

subgraph of a graph G = (V, E): a graph (W, F) where 
ff is a subset of V' and F is a subset of E 
Gi afri (union of G] and GG: the graph (V i U V 2 , E\ u£ ; ) 

where Gi = (Vi, E \) andGj = (Vri.fej) 
adjacency matrix: a matrix representing a graph using the 
adjacency of vertices 

incidence matrix; a matrix, representing a graph using the 
incidence of edges and vertices 
isomorphic simple graphs: the simple graphs Gj ^ 
and (> : - (V : , Ei) are isomorphic if there is 
a one-to-one correspondence/ from Vj to Vri such that 
{/(Vi). ft U)} E £; if and only if {vj, v 2 J E £, for all v ( 
and v. in Vj 

invariant; a. property that isomorphic graphs either both 
have or both do not have 

path from u to v in an undirected graph: a sequence of 
one or inure edges fj, e% ,,., where is associated to 

{at,. a, - 1 1 for / = 0. I. n where Jtp = u and ay = v 

path from u to v in a graph with directed edges: a se¬ 
quence nf otic or more edges c : .where e t is 


associated to (ay, a,*j) for i ~ 0, 1,,.., n where xq = u 
and s,:, i v 

simple path: a path that dues not contain an edge more 
than once 

circuit: a path that begins and ends at the same vertex 
connected graph: an undirected graph with the property 
that there is a path between every pair of vertices in ihe 
graph 

Connected components: the set of connected subgraphs of 
a graph such that no two of these subgraphs have a verte x 
in common 

Euier circuit: a circuit that contains every' edge of a graph 
exactly once 

Euler path: a path that contains every edge nf a graph ex - 
achy once 

Hamilton path: a path xq. xj,,, x a in a simple graph 

G - (V,E) such thatj.ro. _r.,} -- V* and x, ff \. 

for 0 i i < j it n 

Hamilton circuit: a circuit % x\ ..t,, in a simple 

graph such that xq, xi, ..., x n is a Hamilton path 
weighted graph: a graph with numbers assigned to its 
edges 

shortest path problem: the problem (if determining the 
path in a weighted graph such that the sunt of the weights 
of the edges in this path is a minimum over all paths be¬ 
tween specified vertices 

traveling salesman problem: the problem thaL asks for 
the circuit of shortest total length which visits every ver¬ 
tex of the graph exactly once 

planar graph: a graph that can be drawn in the plane with 
no crossings 

regions of a representation of a planar graph: the re¬ 
gions the plane is divided into by the planar representa¬ 
tion of the graph 

elementary subdivision: the removal of an edge {«, ft gf 
an undirected graph and the addition of a new vertex ir 
together with edges {«, w} and {w, v} 
homeomorphic: two undirected graphs are bonieomor- 
phie if they can be obtained from the same graph by a 
sequence of elementary subdivisions 
graph coloring: an assignment of colors to the vertices oi 
a graph so that no two adjacent vertices have the same 
color 

chromatic number: die minimum number of colors 
needed in a coloring of a graph 

RESULTS 

There is an Utiler circuit in a connected multigraph if and 
only if every vertex has even degree. 

There is an Euler path in a connected multigraph if and onlv 
if at most two vertices have odd degree. 

Dijkstra’s algorithm: a procedure for finding the shortest 
path between two vertices in a weighted staph (see paee 
494). 
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Euler's formula: r = c - v + 2 where /, e. and v are the 
number of regions of a planar representation, the number 
of edges, and the number of vertices, respectively, of a 
planar graph. 

Kuratowski's theorem: A graph is nonplanar if and only 


if it contains a subgraph homeomorphic to K .;. or K< . 
(Proof beyond scope of this book.) 

The four color theorem: Every planar graph can be col¬ 
ored using no more than four colors. (Proof for beyond 
the scope of this book’) 


Review Questions 


1. a) Define a simple graph, a multigraph, a pseudo- 

graph, a directed graph, and a directed multigraph, 

b) Use an example to show how each of the types of 
graph in part (at can be used in modeling, For ex¬ 
ample, explain how to mode! different aspects of a 
computer network or airline routes. 

2. Give at least four examples of how graphs are used in 
modeling. 

3. What is the relationship between the sum of the degrees 
of (he vertices in an undirected graph and the number 
of edges in this graph? Explain why this relationship 
holds, 

4. Why must there he an even number of vertices of odd 
degree in an undirected graph? 

5. Whai is the relationship between the sum of the in- 
degrees and the sum of the on [-degrees of the ver¬ 
tices in a directed graph? Explain why this relationship 
holds, 

<i. Describe the following families of graphs, 

a) A’,,, the complete graph on n vortices 

b) K mn . the complete bipartite graph on m and n ver¬ 
tices 

c) r„. the cycle with n vertices 

d) IV-, the wheel of size H 
«-i (> she n-cube 

7. Jlnw many vertices and how many edges are there in 
eadi of die graphs in the families in Question 6? 

8. a) What is a bipartite graph? 

b) Whieh ufthe graphs K„, C n , and Iff are bipartite? 
C) How can you determine whether an undirected 
graph is bipartite? 

9. a) Describe three different methods that can be used 

so represent a graph, 

b) Draw a simple graph with at least five vertices and 
eight edges. Illustrate how it can be represented us¬ 
ing the methods you described in part (a). 

10, a) What does it mean for two simple graphs to be iso¬ 
morphic'i 1 

b) What is meant by an invariant with respect to iso¬ 
morphism for simple graphs 1 ’ Give at least five ex¬ 
amples of such invariants. 

c) Give an example of two graphs that have the same 
numbers of vertices, edges, and degrees of vertices, 
but that arc nor isomorphic. 


d) rs a set of invariants known that can be used to ef¬ 
ficiently determine whether two simple graphs are 
isomorphic? 

11. a) What does it mean for a graph to be connected? 
b) What are the connected components of a graph? 

12. at Explain how an adjacency matrix can be used to 

represent a graph. 

b) How can adjacency matrices be used to determine 
whether a function from the vertex set of a graph C< 
to the vertex set of a graph H is an isomorphism 1 ’ 

e) How can the adjacency matrix of a graph be used to 
determine the number of paths of length r, where r is 
a positive integer, between two vertices of a graph? 

13. a) Define an Euler circuit and an Euler path man undi¬ 

rected graph. 

b) Describe the famous Konigsberg bridge problem 
and explain how to rephrase it in terms of an Eu¬ 
ler circuit. 

c) How can it be determined whether an undirected 
graph has an Euler path? 

d) How can it be determined whether an undirected 
graph has an Euler circuit? 

14. a) Define a Hamilton circuit in a simple graph 

b) Give some properties of a simple graph that imply 
that it does not have a Hamilton circuit. 

15. Give examples of at least two problems that can be 
solved by finding the shortest path in a weighted graph. 

in. a) Describe Dijkstra's algorithm for finding the short¬ 
est path in a weighted graph between two vertices, 
b) Draw a weighted graph w ith at least. 10 vertices and 
20edges. Use Dijkstra’s algorithm to find the short¬ 
est path between two vertices of your choice in the 
graph. 

17. a) What does it mean for a graph to be planar? 
h) Give an example of a nonplanar graph. 

18. a) What is Euler’s formula for planar graphs? 

b) How can Euler's formula for planar graphs be used 
to show r that a simple graph is non-planar? 

19. State Kuratowski’s theorem on die planarity of graphs 
and explain how it characterizes which graphs are pla¬ 
nar. 

20. a) Detine the chromatic number of a graph, 

b) What is the chromatic number of the graph K a when 
n is a positive integer? 
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d Whitt is the chromatic numberof the graph C K when 
n is a positive integer greater than 2'.' 
di What is the chromatic number of the graph K m „ 
when m and n are positive integers'? 


21. State the four color theorem Are there graphs that can 
not be colored with four colors? 

22. Explain how graph coloring can be used in modeling 
Use at least two different examples. 


S uppleme Mary Ex ere i ses 


J. How many edges does a 50-regular graph with J (X) ver¬ 
tices have? 

2. i low many non isomorphic subgraphs does A? have 11 

In Exercises 5-5 determine whether the given pair of graphs 
is isomorphic. 


3. a. u : a-, u j 




% 


The complete m-partite graph K„ x „ m has vertices par¬ 
titioned into m subsets of «j, n >..... n m elements each, and 
vertices are adjacent if and only if they are in different sub¬ 
sets in the partition. 

6. Draw the following, graphs, 

a) K UJ b) K xu c) A'i.v W 
*7. How many vertices and ho'w many edges does the com¬ 
plete w-pariite graph K„ t . nm have? 

*X, a) Prove or disprove that there are always two vertices 
with the same degree in a finite simple graph having 
at least two vertices. 

bl Do the same as in part (a) for finite rmilligraphs. 

LetG - (V] E) be a simple graph. The subgraph induced 
by a subset Wof the vertex set V T is the graph (W. F), where 
the edge set F contains an edge in E if and only if both 
endpoints of this edge are in fV. 
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9. Consider the graph shown in Figure 3 of Section 7,4, 
Find the subgraphs induced by 
a) 

bl {(i, e, g). 
c) {tie, f, g .h). 

10. Eel n be a positive integer. Show that a subgraph in¬ 
duced by a nonempty subset of the vertex set of K„ is 
a complete graph. 



A clique in a simple undirected graph is a complete 
subgraph that is not contained in any larger complete 
subgraph. In Exercises 11-13 find all cliques in the given 
graph, 

li, a h 



12 . 



13. b 



A dominating set of vertices in a simple graph is a set of 
vertices such that every other vertex is adjacent to at least 
one vertex of this set, A dominating set with the least num¬ 
ber of vertices i s called a minimum dominating set. In Ex¬ 
ercises 14- 16 find a minimum dominating set for the given 
graph, 


15. o e 



A simple graph can be used to determine the minimum 
number of queens on a chessboard that control the enure 
chessboard. An ti X n chessboard has iv squares in an 
n X n configuration, A queen in a given position controls 
all squares in the same row, the same column, and on the 
two diagonals containing this square, as illustrated- The ap¬ 
propriate simple graph has >v vertices, one for each square, 
and two vertices are adjacent if a queen in the square repre¬ 
sented by one of the vertices controls the square represented 
by the other vertex. 



17, Construct the simple graph representing the n X n 
chessboard with edges representing the control of 
squares by queens for 
a) n -- 3. b) a 4, 
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18. Explain how the concept of a mi ni mum dominating set 
applies to the problem of determining the minimum 
number of queens controlling a nn x n chessboard, 
**19. Find the minimum number of queens controlling an 
n X n chessboard for 

a) n = 3. b) n = 4, t) n = 5. 

20. Suppose that G, and H\ arc isomorphic and that (T and 
H; are isomorphic. Prove or disprove that G\ U G : and 
Hi U Hi tire isomorphic. 

21. Show that each of the following properties is an invari¬ 
ant that isomorphic simple graphs cither both have nr 
both do not have. 

a) connectedness 

b) the existence of a Hamilton circuit 

c) ihe existence of an Euler circuit 

d) having crossing number C 

e) having n isolated vertices 

f) being bipartite 

22. How can the adjacency matrix of G be found from the 
adjacency matnx of G\ where G is a simple graph? 

23. How many nonisomorphie connected bipartite simple 
graphs are there with four vertices? 

*24, How many mmisomorphic simple connected graphs 
with five vertices are there 
a) with no vertex of degree more than 2? 
hi with chromatic number equal to 4? 
e) that are non planar? 

A directed graph is self-converse if it is isomorphic to its 
converse. 

25, Determine whether the following graphs are self¬ 
converse, 

a) n b 



directed graph is strongly connected. When an orienta¬ 
tion of an undirected graph exists, this graph is called ori- 
entable. In Exercises 27-29 determine whether the given 
simple graph is orientable. 


27. 


28. 


29, 




30. Because traffic is growing heavy in the central part 
of a city, traffic engineers are planning to change all 
the streets, which are currently two-way, into one-way 
streets. Explain how to model this problem. 

*31. Show that a graph is not orientable if it has a cut edge. 

A tournament is a simple directed graph such that if u and 
v are distinct vertices in the graph, exactly one of (w, v) and 
(>’, u) is an edge of the graph. 



26, Show that if the directed graph G is self-converse and 
H is a directed graph isomorphic to G. then H is also 
self-converse. 

An orientation of an undirected simple graph is an 
assignment of directions to its edges so that the resulting 


32. How many different tournaments are there with n ver¬ 
tices? 

33. What is the sum of the in-degree and out-degree of a 
vertex in a tournament? 

*34, Show that every tournament has a Hamilton path. 

35. Given two chickens in a flock, one of them is dominant 
This defines the pecking order of the flock. How can 
a tournament he used to model pecking order? 

36, Suppose that G is a connected multigraph with 2k ver¬ 
tices of odd degree. Show that there exist k subgraphs 
that have G as their union, where each of these Sub¬ 
graphs has an Euler path and where no two of these sub¬ 
graphs have an edge in common. (Hint; Add k edges to 
the graph connecting pairs of vertices of odd degree and 
use an Euler circuit in this larger graph.) 
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*37. Lei G be a simple graph with n vertices. The bandwidth 
o l L O', deno ted by S( G 1, is the mini mum, u v er all permuta¬ 
tions, a-_, ai . a n of the vertices of G, of max{jt - yj a,- 

:uid a , are adjacent}. That is, the bandwidth is the mini¬ 
mum over all listings of the vertices of the maximum dif¬ 
ference in the indices assigned to adjacent vertices. Find 
the hand widths of the following graphs, 

a) K, b) K u c) 

d) A.. ei ft f) ft 

*38. The distance between two distinct vertices Vi and vi 
of a connected simple graph is the length (number of 
edges ) of the shortest path between ty and ra. The ra¬ 
dius of a graph is the minimum over all vertices v of 
the maximum distance from v to another vertex. The 
diameter of a graph is the maximum distance between 
two distinct vertices. Find the radius and diameter of 

a) Aft b) A •. =. c) ft d) ft. 

*39. a) Show that if the diameter of the simple graph G is 
ai least 4, then the diameter of its complement G is 
no more than 2. 

b) Show that if the diameter of the simple graph G is 
at least 3, then the diameter of its complement G is 
no more than 3. 

*40, Suppose that a multigraph has 2m vertices of odd de¬ 
gree. Show that any circuit that contains ever)' edge of 
the graph must contain at least m edges more than once. 

41, Find the second shortest path between die vertices a 
and z in Figure 3 of Section 7.6, 

42, Devise an algorithm for finding the second short¬ 
est path between two vertices in a simple connected 
weighted graph. 

43. Find the shortest path between the vertices a and ; that 
passes through the vertex e in the weighted graph in 
Figure 4 in Section 7.6. 

44. Devise an algorithm for finding the shortest path be¬ 
tween two vertices in a simple connected weighted 
graph that passes through a specified third vertex. 

*45. Show that if G is a simple graph with at least 11 ver¬ 
tices, then either Li or Li, the complement of G, is non- 
plan ar. 

5 A set of vertices in a graph is called independent if no two 
vertices m the set are adjacent. The independence num¬ 
ber of a graph is the maximum number of vertices in an 
independent set of vertices for the graph. 

*46. What is the independence number of 

a) K„? b) ft? c> ft 1 dt A.,/' 


47. Show r that the number of vertices in a simple graph is 
less than or equal to the product of the independence 
number and the chromatic number of the graph. 

48. Show that the chromatic number of a graph is less than 
or equal to v - i + 1, w here v is the number of vertices 
in the graph and i is the independence number of this 
graph. 

49. Suppose that to generate a random simple graph with n 
vertices we first choose a real number p with 0 s p £ 

I. For each of the C(n. 2) pairs of distinct vertices we 
generate a random number a between fl and I. if 0 £ 
x £ p. wc connect these two vertices with an edge; 
otherwise these vertices are not connected, 

a) What is the probability that a graph with m edges 
where 0 £ m -= Cfrt, 2) ts generated? 

b) What is the expected number of edges in a ran¬ 
domly generated graph with n vertices if each edge 
is included with probability p ? 

e) Show that, if p = i/2 then every simple graph with 
n vertices is equally likely to be generated. 

A property retained whenever additional edges are added 
to a simple graph (without adding vertices) is called mono¬ 
tone increasing, and a property that is retained whenever 
edges are removed from a simple graph (without removing 
vertices) is called monotone decreasing. 

50. For each of the following properties, determine 
whether it is monotone increasing and determine 
whether it is monotone decreasing. 

a) The graph G is connected. 

b) The graph G is not connected. 

c) The graph G has an Euler circuit, 

d) Hie graph G has a Hamilton circuit. 

e) The graph G is planar. 

f) The graph G has chromatic number four, 

g) The graph G has radius three, 
lit The graph G has diameter three. 

51. Show that the graph property P is monotone increas¬ 
ing if and only if the graph property Q is monotone 
decreasing where Q is the property of not having 
property P. 

**52. Suppose that P is a monotone increasing property of 
simple graphs. Show that the probability a random 
graph with n vertices has property Pisa monotonic 
nondecreasing function of p, the probability an edge is 
chosen to be in the graph. 


Computer Projects 

WRtTU PROGRAMS WITH THU FOLLOWING INPUT AND OUTPUT. 

I. Given the vertex pairs associated to the edges of an 2. Given Lhe ordered pairs of vertices associated to the 
undirected graph, determine the degree of each vertex. edges of a directed graph, determine the in-degrcc and 
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ouMlegree of each vertex. 

3. Given the list of edges of a simple graph, determine 
whether the graph is bipartite. 

4. Given the vertex pairs associated to the edges of a 
graph, construct an adjacency matrix for the graph. 
(Produce a version that works when loops, multiple 
edges, or directed edges are present.) 

5. Given an adjacency matrix of a graph, list the edges 
of this graph and give the number of times each edge 
appears. 

6 . Given the vertex pairs associated to the edges of 
an undirected graph and the number of times each 
edge appears, construct an incidence matrix for the 
graph. 

7. Given an incidence matrix uf an undirected graph, list 
its edges and give the number of times each edge 
appears. 

8 . Given a positive integer n, generate an undirected 
graph by producing tin adjacency matrix for the graph 
so that all s ini pie graphs are equally likely to be gener¬ 
ated. 

9. Given a positive integer n, generate a directed graph by 
producing an adjacency matrix for the graph so that all 
directed graphs are equally likely to be generated. 

10. Given the lists of edges of two simple graphs with no 
more than six vertices, detennine whether the graphs 
tire isomorphic. 

11. Given an adjacency matrix of a graph anti a positive 
integer n. find the number of paths of length n between 


two vertices. (Produce a version that works for directed 
and undirected graphs.) 

*12. Given the list of edges of a simple graph, determine 
whether it is connected and find the number of con¬ 
nected components if it is not connected. 

13. Given the vertex pairs associated to the edges of a 
mulligraph, determine whether it has an Euler circuit 
and, if not, w hether it has an Euler path. Construct an 
Euler path or circuit if it exists, 

*14. Given the ordered pairs of vertices associated to the 
edges of a directed mulligraph, construct an Euler path 
or Euler circuit, if such a path or circuit exists. 

**15. Given the list of edges of a simple graph, produce a 
Hamilton circuit, or determine that the graph does not 
have such a circuit, 

**16, Given the list of edges of a simple graph, produce a 
Hamilton path, or determine that the graph does not 
have such a path. 

17. Given the list of edges and weights of these edges nf a 
weighted connected simple graph and two vertices in 
this graph, find the length of the shortest path between 
them using Dijkstnvs algorithm. Also, find this path. 

18. Given the list of edges of an undirected graph, find a 
coloring of this graph using the algorithm given in the 
exercise set of Section 7.8. 

19. Given a list of students and the courses that they are 
enrolled in, construe! a schedule of final exams. 

20. Given the distances between pairs of television sta¬ 
tions, assign frequencies to these stations. 


Computations and Explorations 

1 iSl, A GOMPl -RTIOYAI . PROGRAM OR PROGRAMS YOU HAVE. WRITTEN TO DO 
ill POL.LOWING EXERCISES. 


1. Display all the simple graphs with four vertices. 

2. Display a full set of non isomorphic simple graphs with 
six vertices. 

3. Display a lull ser of non isomorphic directed graphs 
with four vertices. 

4. Generate at random 10 different simple graphs each 
with 20 vertices so that each such graph is equally 
likely to be generated. 

5. Construct a Gray code where the code words are bit 
strings of length 6. 

6 . Construct knight's tours on chessboards of various 
sizes. 

7. Determine whether each of the graphs you generated 
in Exercise 4 of this set is planar. If you can, determine 
the thickness of each of the graphs that are not planar. 

8 . Determine whether each of the graphs you generated 
i n Exercise 4 of this set is connected. If a graph is not 
connected, detennine the number of connected compo¬ 
nents nf the graph. 


9. Generate at random simple graphs with 10 vertices. 
Stop when you have constructed one. with an Euler cir¬ 
cuit. Display an Euler circuit in this graph, 

10. Generate at random simple graphs with 10 vertices. 
Stop when you have constructed one with a Hamilton 
circuit. Display a Hamilton circuit in this graph. 

11, Find the chromatic number of each of the graphs you 
generated in Exercise 4 of this set. 

**12. Find the shortest path a traveling salesperson can take 
to visit each of the capitals of the 50 states in the U.S., 
traveling by air between cities in a straight line. 

*13. Estimate the probability that a randomly generated sim¬ 
ple graph with n vertices is connected for each positive 
integer n not exceeding 10 by generating a set of ran¬ 
dom simple graphs and determining whether each is 
connected, 

**14, Work on the problem of determining whether tire cross¬ 
ing number of A’{7.7) is 77, 79, or 81. It is known that 
it equals one of these three values. 
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Writing Projects 

RESPOND TO Ti lE FOLLOWING QUESTIONS WITH ESSAYS USING OUTSIDE SOURCES' 


1. Describe the origins and development of graph the or)’ 
prior to Lhe year 1900. 

2. Discuss the applications of graph theory to the study of 
ecosystems. 

3. Discuss the applications of graph theory to sociology 
and psychology. 

4. Describe algorithms for drawing a graph on paper 
or on a display given the vertices and edges of the 
graph. What considerations arise in drawing a graph 
so that it has the best appearance for understanding 
its properties? 

5. What are some of the capabilities that a software 
tool for inputting, displaying, and manipulating graphs 
should have? Which of these capabilities do available 
tools have? 

6 . Describe some nt the algorithms available for de¬ 
termining whether two graphs are isomorphic and 
the computational complexity of these algorithms, 
What is the most efficient such algorithm currently 

known? 

7. Define de Bruijn sequences and discuss how they arise 
in applications. Explain how de Bruijn sequences can 
be constructed using Filter circuits. 

S. Describe the Chinese Post man Problem and explain 
how to solve this problem. 


9, Describe some of the different conditions that imply 
that a graph has a Hamilton circuit. 

10. Describe some of the strategies and algorithms used to 
solve the traveling salesman problem. 

11. Describe several different algorithms for determining 
whether a graph is planar. What is the computational 
complexity' of each of these algorithms? 

12. In modeling, very large scale integration (VLSI) graphs 
are sometimes embedded in a book, with the vertices 
on the spine and the edges on pages. Define the book 
number of a graph and find the book number of various 
graphs including K, : for n - 3,4,5, and 6. 

13. Discuss the history’ of the four color theorem. 

14. Describe the role computers played in the proof of the 
four color theorem. How can we be sure that a proof 
that relies on a computer is correct? 

15. Describe and compare several different algorithms for 
coloring a graph, in terms of whether they produce a 
coloring with the least number of colors possible and 
in terms of their complexity. 

16. Explain how graph multicolorings can be used in a va¬ 
riety of different models, 

17. Explain how the theory' of random graphs can be used 
in nonconstructive existence proofs of graphs with cer¬ 
tain properties. 
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A connected graph that contains no simple circuits is called a tree. Trees were 
used as long ago as 1857, when the English mathematician Arthur Cayley used 
them to count certain types of chemical compounds, Since that time, trees have 
been employed to solve problems in a wide variety of disciplines, as the examples in 
this chapter will show. 

Trees are particularly useful in computer science. For instance, trees are employed 
to construct efficient algorithms for locating items in a list. They are used to construct 
networks with the least expensive set of telephone lines linking distributed computers. 
Trees can be used to construct efficient codes for storing and transmitting data. Trees 
can model procedures that arc carried out using a sequence of" decisions. This makes 
trees valuable in the study of sorting algorithms. 


8J_ _ 

Introduction to Trees 

- A genealogical chart of the Bemoullis. a famous family of Swiss mathematicians, is 

neft shown in Figure 1. Such a chart is also called a family tree. A family tree is a graph 
where the vertices represent family members and the edges represent parent-child 
relationships. The undirected graph that represents a genealogical chart is an example 
of a special type of graph called a tret. 


DEFINITION 1. A tree is a connected undirected graph with no simple circuits. 

Since a tree cannot have a simple circuit, a tree cannot contain multiple edges or leaps, 
Therefore any free must be a simple graph. 


EXAMPLE 1 Which pi the graphs shown in Figure 2 are trees'? 

Solid ion: Oj and O'; are trees, since both are connected graphs with no simple circuits. 
Gs is not it tree because r. b, a. d, e is a simple circuit in this graph. Finally, G\ is not 
a tree since it is not connected. ■ 


Any connected graph that contains no simple circuits is a tree. What about graphs con¬ 
taining no simple circuits that arc not necessarily connected'? These graphs arc called 
52S 
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THEOREM 1 



Jacob I 
(1654-1705) 


Nikolaus 

(1623-1708) 


Nikolaus 

{1662-17161 


Johann I 
(1667-1748) 


Nikolaus I 
(1687-1759) 


Nikolaus II 
(1695-17 26) 


Daniel 

(1700-1782) 


Johann II 
(1710-1790) 


FIGURE 1 The Bernoulli Family of Mathematicians. 


Johann III Jackobll 
(1746-1807) (1759-1789) 


forests and have the property that each of their connected components is a tree. Fig¬ 
ure 3 displays a forest. 

Trees are often defined as undirected graphs with the property that there is a unique 
simple path between every pair of vertices. The following theorem shows that this al¬ 
ternative definition is equivalent to our definition. 

An undirected graph is a tree if and only if there is a unique simple path between 
any two of its vertices. 

Proof: First assume that T is a tree. Then T is a connected graph with no simple circuits. 
Let jt and _v be two vertices of T. Since T is connected, by Theorem 1 of Section 7.4 there 
i s a simple path between x andy. Moreover, this path must be unique, for if there were 
a second such path, the path formed by combining the first path from x to y followed 
by the path from y to x obtained by reversing the order of the second path from x to 
y would form a circuit. This implies, using Exercise 35 of Section 7.4, that there is a 



a i (h r;, c, 

FIGURE 2 Gi and Gi Are Trees; G, and Gj Are Not 
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This is one graph with three connected components 



HGURE 3 Example of a Forest 


simple circuit in T. Hence, there is a unique simple path between tiny two vertices of a 
tree. 

Now assume that there is a unique simple path between any two vertices of a graph 
T. Then T is connected, since there is a path between any two of its vertices. Further¬ 
more. T can have no simple circuits. To see that this is true, suppose 7 had a simple 
circuit that contained the vertices x and y. Then there would be two simple paths be¬ 
tween x and y, since the simple circuit is made up of a simple path from x to y and a 
second simple path from v tox. Hence, a graph with a unique simple path between any 
two vertices is a tree. □ 

In many applications of trees a particular vertex of a tree is designated as the root. 
Once we specify a root, we can assign a direction to each edge as follows. Since there is 
a unique path from the root to each vertex of the graph (from Theorem 1), we direct each 
edge away from the root. Thus, a tree together with its root produces a directed graph 
called a rooted tree. We can change an unrooted tree into a rooted tree by choosing any 
vertex as the root. Note that different choices of the root produce different rooted trees. 
For instance, Figure 4 displays the rooted trees formed by designating a to be the root 
and c to be the root, respectively, in the tree T. We usually draw a rooted tree with its 
root at the top of the graph. The arrows indicating the directions of the edges in a rooted 
tree can be omitted, since the choice of root determines the directions of the edges. 

The terminology for trees has botanical and genealogical origins. Suppose that T 
is a rooted tree. If v is a vertex in 7 other than the root, the parent of v is the unique 



T With root a With root c 

FIGURE 4 A Tree and Rooted Trees f ormed by Desi g n a ting Two Roots. 
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EXAMPLE 2 


vertex, u such that there is a directed edge front u to v (the reader should show that such 
a vertex is unique). When a is the parent of v, v is called a child of u. Vertices with the 
same parent are called siblings. The ancestors of a vertex other than the root are the 
vertices in the path from the root to this vertex, excludi ng the vertex itself and including 
the root {that is, its parent, its parent's parent, and so on, until the root is reached). The 
descendants of a vertex v are those vertices that have v as an ancestor. A vertex of a 
tree is called a leaf if it has no children. Vertices that have children are called internal 
vertices. The root is an internal vertex unless it is the only vertex in the graph, in which 
case it is a leaf. 

If a is a vertex in a tree, the subtree with a as its root is the subgraph of the tree 
consisting of a and its descendants and all edges incident to these descendants. 


In the rooted tree T ( with root a) shown in Figure 5, find the parent of c, the children of 
g, the siblings of h, all ancestors of e, all descendants of h, all internal vertices, and all 
leaves. What is the subtree rooted at g? 


Solution: The parent of c is b. The children of g are h, i, and j. The siblings of k are 
i and j. The ancestors of e are c, b, and a. The descendants of b are c, d, and e. The 
interna) vertices are a, b, c, g t h, and /. The leaves are d, e.f, i, k, (, and m. The subtree 
rooted at g is shown in Figure 6. ■ 


Rooted trees with the property that all of their internal vertices have the same num¬ 
ber of children are used in many different applications. Latei in this chapter we will 
use such trees to study problems involving searching, sorting, and coding. 


DEFINITION 2. A rooted tree is called an m-ary tree if every internal vertex has 
no more than m children. The tree is called a fad I m-ary tree if every internal vertex 
has exactly m children. An m-ary tree with m = 2 is called a binary tree. 


u T 



g 



FIGURE b The Subtree Rooted At g. 
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EXAMPLE 3 


EXAMPLE 4 



Are the rooted trees in Figure 7 full wi-ary trees for some positive integer ml 

Solution: T\ is a full binary tree since each of its internal vertices has two children. 
Ti is a full 3-ary tree since each of its internal vertices has three children. In 73 each 
interna! vertex has five children, so T\ is a full 5-ary tree. T\ is not a full m-ary tree 
for any m since some of its internal vertices have two children and others have three 
children. M 


An ordered rooted tree is a rooted tree where the children of each internal vertex 
are ordered. Ordered rooted trees are drawn so that the children of each internal vertex 
are shown in order from left to right. Note that a representation of a rooted tree in the 
conventional way determines an ordering for its edges. We will use such orderings of 
edges in drawings without explicitly mentioning that we are considering a rooted tree 
to be ordered. 

In an ordered binary' tree, if an internal vertex has two children, the first child is 
called the left child and the second child is called the right child. The tree rooted at the 
left child of a vertex is called the left subtree of this vertex, and the tree rooted at the 
right child of a vertex is called the right subtree of the vertex. The reader should note 
that for some appl icalions every vertex of a binary tree, other than the root, is designated 
as a right or a left child of its parent. This is done even when some vertices have only 
one child. We will make such designations whenever it is necessary, but not otherwise, 


What are the left and right children of d in the binary tree T shown in Figure 8(a) (where 
the order is that implied by the drawing)? What are the left and right subtrees of c? 
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EXAMPLE 5 



FIGURE S A Binary Tree T and Left and Right Subtrees of the Vertex c. 


Solution: The left child of d is f and the right child is g. We show the left and right 
subtrees of c in Figures 8(b) and 8(c), respectively. ■ 


Just as in the case of graphs, there is no standard terminology used to describe trees, 
rooted trees, ordered rooted trees, and binary trees. This nonstandard terminology oc¬ 
curs since trees are used extensively throughout computer science, which is a relatively 
young field. The reader should carefully cheek meanings given to terms dealing with 
trees whenever they occur. 


TREES AS MODELS 


"frees are used as models in such diverse areas as computer science, chemistry, geology, 
botany, and psychology. We will describe a variety of such models based on trees, 


Saturated Hydrocarbons and Trees Graphs can be used to represent molecules, where 
atoms are represented by vertices and bonds between them by edges. The English math¬ 
ematician Arthur Cayley discovered trees in 1857 when he was trying to enumerate the 
isomers of compounds of the form . which are called saturated hydrocarbons. 

In graph models of saturated hydrocarbons, each carbon atom is represented by a 
vertex of degree 4, and each hydrogen atom is represented by a vertex of degree 1. 


Arthur Cayley (1821-1895). Arthur Cayley, the son of a merchant, displayed his mathematical talents 
at an early age with amazing skill in numerical calculations. Cayley entered Trinity College, Cambridge, 
when he was 17. While in college he developed a passion for reading novels, Cayley excelled at Cambridge 
and was elected to a 3-year 'appointment as Fellow of Trinity and assistant tutor. During this time Cayley 
began his study of rt-dimensional geometry and made a variety of contributions to geometry and to analysts. 
He also developed an interest in mountaineering, which he enjoyed during vacations in Switzerland. Since 
no position as a mathematician was available to him, Cayley left Cambridge, entering the legal profession 
and gaining admittance to the bar in 1849. Although Cayley limited his legal work to be able to continue 
his mathematics research, he developed a reputation as a legal specialist. During his legal career he was 
able to write more than 300 mathematical papers. In 1863 Cambridge University established a new post in 
mathematics and offered it to Cayley, He look this job, even though it paid less money than he made as a 
lawyer. 
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I 

\ 
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H — C —H 

I 

H 


Butane Isobutane 

FIf.CRE 9 The Two Isomers of Butane, 


There are 3 n + 2 vertices in a graph representing a compound of the form CsH^+j. The 
number of edges in such a graph is half the sum of the degrees of the vertices. Hence, 
there are (4n + 2n + 2)12 - 3« + 1 edges in this graph. Since the graph is connected 
and the number of edges is one less than the number of vertices, it must be a tree (see 
Exercise 9 at the end of this section). 

The nonisomorphic trees with n vertices of degree 4 and 2 n+2 of degree l represent 
the different isomers of For instance, when n - 4, there are exactly two 

nonisomorphic trees of this type (the reader should verify this). Hence, there are exactly 
two different isomers of CaHjo- Their structures are displayed in Figure 9. These two 
isomers are called butane and isobutane. ■ 


EXAMPLE $ Representi ng Organizations The structure of a large organization can be modeled 

using a rooted tree. Each vertex, in this tree represents a position in the organiza¬ 
tion. An edge from one vertex to another indicates that the person represented by the 
initial vertex is the (direct) boss of the person represented by the terminal vertex. 
The graph shown in Figure 10 displays such a tree. In the organization represented 
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EXAMPLE 7 


EXAMPLE H 


The root is the root directory / 
internal vertices are directories 
Leaves are iiles 



hi n rjio spool Is mail who junk 



ed nroft' vi khr opr uucp 


El r inter Ilk 

F1GURF, U A Computer File System. 


by this tree, the Director of Hardware Development works directly for the Vice Presi¬ 
dent of R&D. ■ 


Computer File Systems Files in computer memory can be organized into directories. 
A directory can contain both files and subdirectories. The root directory’ contains the 
entire tile system. Thus, a file system may be represented by a rooted tree, where 
the root represents the root directory', internal vertices represent subdirectories, and 
leaves represent ordinary files or empty directories. One such file system is shown m 
Figure 11. In this system, the file khr is in the directory rje. ■ 


Tree-Connected Parallel Processors In Example 13 of Section 7.2 we described sev¬ 
eral interconnection networks for parallel processing. A tree-connected network is 
another important way to interconnect processors. The graph representing such a net¬ 
work is a complete binary tree. Such a network interconnects n = 2* - I proces¬ 
sors, where k is a positive integer, A processor represented by the vertex r that is 
not a root or a leaf has 3 two-way connections—one to the processor represented by 
the parent of v and two to the processors represented by the two children of v. The 
processor represented by the root has 2 two-way connections to the processors repre¬ 
sented by its two children. A processor represented by a leaf v has a single two-way 
connection to the parent of v. We display a tree-connected network with seven proces¬ 
sors in Figure 12, 

We will illustrate how a tree-connected network cam be used for parallel compu¬ 
tation. In particular, we will show how the processors in Figure 12 can be used to add 
eight numbers, using three steps. In the first step, we add ay and x 2 using P A , ay and x A 
using P 5 , * 5 and ,v fi using P 6 , and ,v 7 and x 8 using P 7 , In the second step, we add x, + x 2 
and xj +■ a 4 using Pi and x$ + Xf, and x 2 + jtg using Pi. Finally, in the third step, we 
add x i + x% + jti + r 4 and ay + f x 7 + a s using P\. The three steps used to add eight 
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FIGURE 12 A Tree-Connected Network of Seven Processors. 


numbers compares favorably to the seven steps required to add eight numbers serially, 
where the steps are the addition of one number to the sum of the previous numbers in 
the list. ■ 

PROPERTIES OF TREES 

We will often need results relating the numbers of edges and vertices of various types 
in trees. 

A tree with n vertices has n - 1 edges. 

Proof; Choose the vertex r as the root of the tree. We set up a one-to-one correspon¬ 
dence between the edges and the vertices other than r by associating the terminal vertex 
of an edge to that edge. Since there are n - 1 vertices other than r, there are n - ) edges 
in the tree. □ 

The number of vertices in a full m-ary tree with a specified number of interna! 
vertices is determined, as the following theorem shows. As in Theorem 2, we will use 
n to denote the number of vertices in a tree. 

A foil m- ary tree with i internal vertices contains n - mi + 1 vertices. 

Proof: Every vertex, except the root, is the child of an internal vertex. Since each of 
the f internal vertices has m children, there are mi vertices in the tree other than the root. 
Therefore, the tree contains n = mi -t 1 vertices. □ 

Suppose that T is a full m-ary tree. Let i be the number of internal vertices and i the 
number of leaves in this tree. Once one of n, i and / is known, the other two quantities 
tire determined. How to find the other two quantities from the one that is known is given 
in the following theorem. 

A foil m-ary tree with 

(i) n vertices has i = (n - l)/m internal vertices and l = [im - l)n + l]/m 
leaves, 

Hi) i internal vertices has n ~ mi + 1 vertices and ! = (m - I)i + 3 leaves, 

(Hi) /leaveshas n - (ml- 1 )i{m - 1 )verticesandi = (l- 1 )i(m - 1 )internal 
vertices. 
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EXAMPLE *) 


EXAMPLE 10 



h 

FIGURE !3 A Rooted 
Tree. 

EXAMPLE 11 


Proof: Let n represent the number of vertices, i the number of internal vertices, and l 
the number of leaves. The three parts of the theorem can all be proved using the equality 
given in Theorem 3, that is, n - mi + 1, together with the equality n = l + i, which 
is true because each vertex is either a leaf or an internal vertex. We will prove part (i) 
here. The proofs of parts (ii) and (iii) are left as exercises for the reader. 

Solving for i in n ~ mi + 1 gives i = (n— 1 )lm. Then inserting this expression for 
i into the equation n = l + i shows that l — n — i - n- (n - 1 )/m = {(m — 1 )n + 11 fm. 

□ 


The following example illustrates how Theorem 4 can be used. 


Suppose that someone starts a chain letter. Each person who receives the letter is asked 
to send it on to four other people. Some people do this, but others do not send any letters. 
How many people have seen the letter, including the first person, if no one receives more 
than one letter and if the chain letter ends after there have been 100 people w ho read it 
but did not send it out? How many people sent out the letter? 

Solution: The chain letter can be represented using a 4-ary tree. The internal vertices 
correspond to people who sent out the letter, and the leaves correspond to people who 
did not send it out. Since 100 people did not send out the letter, the number of leaves 
in this rooted tree is l = 100. Hence, part (iii) of Theorem 4 show's that the number of 
people who have seen the letter is n = {4 ‘ 100 - l)/(4 ~ 1 ) = 133 . Also, the number 
of internal vertices is 1 33 - 100 = 33 , so that 33 people sent out the letter. ■ 


It is often desirable to use rooted trees that arc “balanced" so that the subtrees at 
each vertex contain paths of approximately the same length. Some definitions will make 
this concept clear. The level of a vertex r in a rooted tree is the length of the unique 
path from the root to this vertex. The level of the root is defined to be zero. The height 
of a rooted tree is the maximum of the levels of vertices. In other words, the height of 
a rooted tree is the length of the longest path from the root to any vertex. 


Find the level of each vertex in the rooted tree shown in Figure 13. What is the height 
of this tree? 

Solution: The root a is at level 0. Vertices h, j, and k are at level 1. Vertices c, e, f and 
/ are at level 2. Vertices d , g, i, m, and n are at level 3. Finally, vertex h is at level 4, 
Since the largest level of any vertex is 4, this tree has height 4. ■ 


A rooted m-ary tree of height h is balanced if all leaves are at levels h or h -- I 


Which of the rooted trees shown in Figure 14 are balanced? 

Solution: T\ is balanced, since all its leaves are at levels 3 and 4. However, T 2 is not 
balanced, since it has leaves at levels 2, 3. and 4. Finally, 7j is balanced, since all its 
leaves are at level 3. ■ 
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FIGURE 14 Some Rooted Trees. 


The following results relate the height and the number of leaves in m-ary trees. 

There are at most rn h leaves in an m-ary tree of height h. 

Proof: The proof uses mathematical induction on the height. First, consider m-ary trees 
of height 1. These trees consist of a root with no more than m children, each of which is 
a leaf. Hence there are no more than m ] - m leaves in an m-ary tree of height 1. This 
is the basic step of the inductive argument. 

Now' assume that the result is true for all m-ary trees of height less than h; this is 
the inductive hypothesis. Let 7" be an m-ary tree of height h , The leaves of T are the 
leaves of the subtrees of T obtained by deleting the edges from the root to each of the 
vertices at level 1, as shown in Figure 15. 

Each of these subtrees has height less than or equal to k - 1. So by the inductive 
hypothesis, each of these rooted trees has at most m h ~ 1 leaves. Since there are at most m 
such subtrees, each with a maximum of m h leaves, there are at most m - m h ~ 1 = m h 
leaves in the rooted tree. This finishes the inductive argument. □ 
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COROLLARY 1 If an »t-ary tree of height h has / leaves, then h a (log*/]: If the «-aiy tree is full 
and balanced, then ft = [l.og M /]. (We are using the celling function here. Recall that 
f x] is the smallest integer greater th an or equal to x) 

Proof: We know that / £ nf from Theorem 5. Taking logarithms to the base m shows 
that log OT / £ h. Since h is an integer, we have h - flog m /]. Now suppose that the tree 
is balanced. Then each leaf is at level h or h 1, and since the height, is ft, there is at least 
one leaf at level h. It follows that there must be more than m h ~ 1 leaves (see Exercise 24 
at the end of this section). Since l < m h , we have m h ~ ] < l < m h . Taking logarithms 
to the base m in this inequality gives ft ~ 1 < log m / < ft. Hence, h - flog m /]. □ 


Exercises 


1. Which of the following graphs arc trees? 



2. Answer the following questions about the rooted tree 
illustrated. 


[J 



a) Which vertex is the root? 

b) Which vertices are internal ? 

c) Which vertices are leaves? 

d) Which vertices arc children of i? 


e) Which vertex is the parent of hi 

f) Which vertices are siblings of ol 

g) \v7iich vertices are ancestors of ml 

h) Which vertices are descendants of ft? 

3. Is the rooted tree in Exercise 2 a full /n-ary tree for some 
positive integer ml 

4. What is the level of each vertex of the tree in Exer¬ 
cise 2? 

5. Draw the subtree of the tree in Exercise 2 that is root¬ 
ed at 

a) a. 
b> c. 
c) e. 

*6, How many nonisomorphic unrooted trees are there with 
n vertices if 

a) n = 3? 

b) n = 4? 

c) n 5? 

*7. Answer the same question as that given in Exercise 
6 for rooted trees (using isomorphism for directed 
graphs). 

*8. Show that a simple graph is a tree if and only if it is 
connected, but the deletion of any of its edges produces 
a graph that is not connected. 

*9, Let G be a simple graph with n vertices. Show that G 
is a tree if and only if G is connected and has n - I 
edges 

10. Which complete bipartite graphs A?„ r „, where m and n 
are positive integers, are trees? 

11. How many edges does a tree with 10,000 vertices 
have? 

12. How many vertices does a full 5-ary tree with 100 in¬ 
ternal vertices have? 

13. How r many edges does a full binary tree with 1000 in¬ 
ternal vertices have? 

14. How many leaves does a full 3-ary tree with 100 ver¬ 
tices have? 
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15, Suppose 1000 people enter a chess tournament. Use a 
rooted tree model of the tournament to determine how 
many games must be played to determine a champion, 
if a player is eliminated after one loss and games are 
played until only one entrant has not lost. (Assume 
there arc no ties.) 

16, A chain letter starts when a person sends a letter to live 
others. Each person who receives the letter cither sends 
it to five other people who have never received it 
or does not send it to anyone. Suppose that 10,000 peo¬ 
ple send out the letter before the chain ends and 
that no one receives more than one letter, How many 
people receive the letter, and how many do not send it 
out? 

17, A chain letter starts with a person sending a letter out 
to 10 others. Each person is asked to send the let¬ 
ter out to 10 others, find each letter contains a list of 
the previous 6 people in the chain. Unless there are 
fewer than six names in the list, each person sends 
one dollar to the first person in this list, removes the 
name of this person from the list, moves up each of 
the other five names one position, and inserts his or 
her name at the end of this list. If no person breaks 
the chain and no one receives more than one letter, 
how much money will a person in the chain ultimately 
receive? 

*18, Either draw' a full m- ary tree with 76 leaves and height 
3 where m is a positive integer or show that no such 
tree exists, 

*19. Either draw a full m-ary tree with 84 leaves and height 
3 where m is a positive integer or show that no such 
tree exists. 

*20. A full m-ary tree T has 81 leaves and height 4. 

a) Give the upper and lower bounds form. 

b) What ts m if T is also balanced'? 

A complete m-ary tree is a full m-ary tree where e very leaf 

is at the same level. 

21. Construct a complete binary tree of height 4 and a com¬ 
plete 3-ary tree of height 3. 

22. How many vertices and how many leaves does a com¬ 
plete m-ary tree of height h have? 

23. Prove 

a) part (ii) of Theorem 4. 

b) part (iii) of Theorem 4. 

r? 24. Show that a full m-ary balanced tree of height h has 
more than m h ~ 1 leaves. 

25. How many edges arc there in a forest of (trees contain¬ 
ing a total of n vertices'.' 

26, Explain how a tree can he used to represent the table of 
contents of a book organized into chapters, where each 
chapter is organized into sections, and each section is 
organized into subsections. 


27, How many different isomers do the following satu¬ 
rated hydrocarbons have? 

a) C } H S b) C 5 Hj 2 c) C 6 Hi4 

28, What does each of the following represent in an 
organizational tree? 

a) the parent of a vertex 

b) a child of a vertex 

c) a sibling of a vertex 

di the ancestors of a vertex 
e) the descendants of a vertex 
f} the level of a vertex 
g) the height of the tree 

29, Answer the same questions as those given m Exercise 
28 for a rooted tree representing a computer file system. 

30, a) Draw the complete binary'tree with 15 vertices that 

represents a tree-connected network of 15 proces¬ 
sors, 

b) Show how 16 numbers can be added using the 15 
processors in part (a) using four steps. 

31, Let n be a power of 2. Show that n numbers can he 
added in log n steps using a tree-connected network of 
n - 1. processors. 

*32, A labeled tree is a tree where each vertex is assigned 
a label. Two labeled trees are considered isomorphic 
when there is an isomorphism between them that pre¬ 
serves the labels of vertices. How many nonisomorphic 
trees are there with three vertices labeled with differ¬ 
ent integers from the set {1.2,3}? How many noniso¬ 
morphic trees are there with four vertices labeled with 
different integers from the set {1,2,3,4)? 

The eccentricity of a vertex in an unrooted tree is the length 
of the longest simple path beginning at this vertex. A ver¬ 
tex is called a center if no vertex in the tree has smaller 
eccentricity' than this vertex. In Exercises 33-35 find every 
vertex that is a center in the given tree. 

33. “ 



34, a el f 



i j k 
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36. Show that a center should be chosen as the root 
to produce a rooted tree of minima] height from an 
unrooted tree. 

*37. Show that a tree has either one center or two centers 
that are adjacent. 


38, Show that every tree can be colored using two colors 

The rooted Fibonacci trees T„ are defined recursively in 
the following way. 7j and TV are both the rooted tree con¬ 
sisting of a single vertex, and for n = 3,4.the routed 

tree T n is constructed from a root with TV, as its left sub¬ 
tree arid TV 2 as its right subtree. 

39, Draw' the first seven rooted Fibonacci trees. 

*40. How many vertices, leaves, and internal vertices does 
the rooted Fibonacci tree T„ have, where n is a positive 
integer? What is its height? 


8.2 _ 

Applications of Trees 

INTRODUCTION 

We will discuss three problems that can be studied using trees. The first problem is: 
How should items in a list be stored so that an item can he easily located? The second 
problem is: What series of decisions should be made to line! an object with a certain 
property in a collection of objects of a certain type? The third problem is: How should 
a set of characters be efficiently coded by bit strings? 

BINARY SEARCH TREES 


Searching for items in a list is one of the most important tasks that arises in computer 
web science. Our primary goal is to implement a searching algorithm that finds items effi¬ 
ciently when the items are totally ordered. This can be accomplished through the use 
of a binary search tree, which is a binary tree in which each child of a vertex is des¬ 
ignated as a right or left child, no vertex has more than one right child or left child, and 
each vertex is labeled with a key. which is one of the items. Furthermore, vertices are 
assigned keys so that the key of a vertex is both larger than the keys of all vertices in 
its left subtree and smaller than the keys of all vertices in its right subtree. 

The following recursi ve procedure is used to form the binary' search tree for a 1 i st o 1 
items. Start with a tree containing just one vertex, namely, the root. The first item in the 
list is assigned as the key of the root. To add a new item, first compare it with the keys 
of vertices already in the tree, starting at the root and moving to the left if the item is 
less than the key of the respective vertex if this vertex has a left child, or moving to the 
right if the item is greater than the key of the respective vertex if this vertex has a right 
child. When the item is less than the respective vertex and this vertex has no left child, 
then a new' vertex with this item as its key is inserted as a new left child. Similarly, 
when the item is greater than the respective vertex and this vertex has no right child, 
then a new vertex with this item as its key is inserted as a new right child. We illustrate 
this procedure with the following example. 


EXAMPLE 1 Form a binary search tree for the words mathematics, physics, geography, zoology, 

meteorology, geology, psychology, and chemistry (using alphabetical order). 
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\ 

Physios 


Phvsin > Mmhcnuukii 


Geography Physics 


Cruography < Maihenaiics 


Geography 


McttSOr&Ii'uv Zwlugy 


Melci'nilo! 1 .) > Nhirhi-nianc 
Meleemiei'v .. Phy^ies 


Mathematics 

Geography/x, 

*’V ^Physics 

Geology \ / "V Zonlosv 

Meteorology 


Mathematic 
I Geography, 



Geag. 



Physics 


Zoology 


Zoology > Mathematics 
Zoology > Phvsics 


Zoology 


Geology 

Genloas 


Malhciiiatrcs 
• G cog rap hy 


Psychology 

Psychology > Malhemalics 
Psychology > Physics 
Psychology <. Zoology 


Meteorology Psychology 


Chi;[insirs < Mathematics 
Chemistry < Gcug 


ithemalK's i 
□graphy 


FiGUUK I Constructing a Binary Starch Tree. 


Solution: Figure I displays the steps used to construct this binary search tree. The 
word mathematics is the key of the root. Since physics comes after mathematics (in 
alphabetical order), add a right child of the root with key physics. Since geography 
comes before mathematics, add a left child of the root with key geography. Next, add a 
right child of the vertex with key physics, and assign it the key zoology, since zoology 
comes after mathematics and after physics. Similarly, add a left child of the vertex with 
key physics and assign this new vertex the key meteorology. Add a right child of the 
vertex with key geography and assign this new vertex the key geology. Add a left child 
of the vertex with key zoology and assign it the key psychology. Add a left child of the 
vertex with key geography and assign it the key chemistry. (The reader should work 
through all the comparisons needed at each step,) ■ 


To locate an item we try to add it to a binary search tree. We locate it if it is present. 
Algorithm 1 gives pseudocode for locating an item in a binary search tree and adding a 
new vertex w ith this item as its key if the item is not found. Algorithm 1 locates x if it 
is already the key of a vertex. When x is not a key, a new vertex with key x is added to 
the tree. In the pseudocode, v is the vertex that has x as its key, and labeliv) represents 
the key of vertex v. 

We will now determine the computational complexity of this procedure. Suppose 
we have a binary' search tree T for a list of n items. We can form a full binary- tree V 
from T by adding unlabeled vertices whenever necessary so that every- vertex with a 
key has two children. This is illustrated in Figure 2. Once we have done this, w-e can 
easily locate or add a new- item as a key without adding a vertex. 

The most comparisons needed to add a new item is the length of the longest path 
in U from the root to a leaf. The interna! vertices of U are the vertices of T. It follows 
that U has n internal vertices. We catt now use part (ii) of Theorem 4 in Section 8.1 
to conclude that U has n 4- I leaves. Using Corollary- 1 of Section 8.1, we see that the 
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algorithm i Binary Search Tree Algorithm. 

procedure insertion(T: binary' search tree, x: item) 
v : - root of T 

{a vertex not present in T has the value nut!) 
while v # null and labeliv) f 6 x 

begin 

if x < lahtil v) then 

if left child of v t 4 null then v : = left child of v 

else add new vertex as a left child of v and set v : = null 

else 

if right child of v t 4 null then v : - right child of u 

else add new; vertex as a right child of v to T and set v ; = null 

end 

if root of T ~ null then add a vertex r to the tree and label it with .1 
else if labeliv) ^ x then label new vertex with x 
{v = location of ,v} 




Unlabeled vertices circled 

_ r _ u _ 

FIGURE 2 Adding llnlabeled Vertices to Make a Binary Search Tree Full. 


height of U is greater than or equal to h = [login + 1)[. Consequently, it is necessary to 
perform at least [login + [)] comparisons to add some item. Note that if U is balanced, 
its height is [logI '? + l)] (from Corollary' 1 of Section 8.1). Thus, if a binary search tree 
is balanced, locating or adding an item requires no more than [login + 1 )] comparisons. 
A binary search tree can become unbalanced as items are added to it. Since balanced 
binary search trees give optimal worst-ease complexity' for binary searching, algorithms 
have been devised that rebalance binary' search trees as items are added. The interested 
reader can consult references on data structures for the description of such algorithms. 


DECISION TREES 


Rooted trees can be used to model problems in which a series of decisions leads to a 
solution. For instance, a binary search tree can be used to locate items based on a series 
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of comparisons, where each comparison tells us whether we have located the item, or 
wgfy whether we should go right or left in a subtree. A rooted tree in which each internal 
vertex corresponds to a decision, with a subtree at these vertices for each possible out¬ 
come of the decision, is called a decision tree. The possible solutions of the problem 
correspond to the paths to the leaves of this rooted tree. The next example illustrates an 
application of decision trees. 

EXAMPLE 2 Suppose there are seven coins, all with the same weight, and a counterfeit coin that 

weighs less than the others. How many weighings are necessary using a balance scale 
to determine which of the eight coins is the counterfeit one? Give an algorithm for 
finding this counterfeit coin. 

Solution: There are three possibilities for each weighing on a balance scale. The two 
pans can have equal weight, the first pan can be heavier, or the second pan can be heav¬ 
ier. Consequently, the decision tree for the sequence of weighings is a 3-ary 1 tree. There 
are at least eight leaves in the decision tree since there are eight possible outcomes 
( since each of the eight coins can be the counterfeit lighter coin ), and each possible out¬ 
come must be represented by at least one leaf. The largest number of weighings needed 
to determine the counterfeit coin is the height of the decision tree. From Corollary' I of 
Section 8.1 it follows that the height of the decision tree is at least flog 3 8] = 2. Hence, 
at least two weighings are needed. 

It is possible to determine the counterfeit coin using two weighings. The decision 
tree that illustrates how this is done is shown in Figure 3. ■ 


In Section 4 of this chapter we will study sorting algorithms using decision trees. 


PREFIX CODES 

Consider the problem of using bit strings to encode the letters of the English alphabet 
(where no distinction is made between lowercase and uppercase letters). We can repre¬ 
sent each letter with a bit string of length five, since- there are only 26 letters and there 
are 32 bit strings of length five. The total number of bits used to encode data is five 
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FIGURE 3 A Decision Tree for Locating a Counterfeit Coin. 


PDF created with pdfFactory trial version www.pdffactorv.com 























8,2 Applications of Trws 545 


times the number of characters in the test when each character is encoded with five 
bits. Is it possible to find a coding scheme of these letters so that, when data are coded, 
fewer bits are used? We can Save memory and reduce transmittal time if this can be 
done. 

Consider using bit strings of different lengths to encode letters. Letters that occur 
more frequently should be encoded using short bit strings, and longer bit strings should 
be used to encode rarely occurring letters. When letters are encoded using varying num¬ 
bers of bits, some method must be used to determine where the bits for each character 
start and end. For instance, if e were encoded with 0, a with ], and t with (11, then the 
bit string 0101 could correspond to ecu, tea, eaea, or it. 

To ensure that no bit string corresponds to more than one sequence of letters, the 
bit string for a letter must never occur as the first part of the bit siring for another letter. 
Codes with this property are called prefix codes. For instance, the encoding of e as 0, 
ti as 10, and t as 11 is a prefix code. A word can be recovered from the unique bit string 
that encodes its letters. For example, the string 10110 is the encoding of ate. To see 
this, note that the initial 1 does not represent a character, but 10 does represent a (and 
could not be the first pan of the bit string of another letter). Then, the next 1 does not 
represent a character, but 11 does represent t. The final bit, 0, represents e. 

A prefix code can be represented using a binary tree, where the characters are the 
labels of the leaves jn the tree. The edges of the tree are labeled so that an edge leading 
to a left child is assigned a 0 and an edge leading to a right child is assigned a 1. The 
bit string used to encode a character is the sequence of labels of the edges m the unique 
path from the root to the leaf that has this character as its label. For instance, the tree in 
Figure 4 represents the encoding of e by 0, a by 10, / by 110, n by 1110. and x by Jill. 

The tree representing a code can be used to decode a bit string. For instance, con¬ 
sider the word encoded by 11111011100 using the code in Figure 4, This bit string can 
be decoded by starting at the root, using the sequence of bits to form a path that stops 
when a leaf is reached. Each 0 bit takes the path down the edge leading to the left child 
of the last vertex in the path, and each 1 bit corresponds to the right child of this vertex. 
Consequently, the initial 1111 corresponds to the path starting at the root, going right 
tour times, leading to a leaf in the graph that has s as its label, since the string 1111 is 
the code for s. Continuing with the fifth bit, we reach a leaf next after going right then 
left, when the vertex labeled with a, which is encoded by 10, is visited. Starting with 
the seventh bit, we reach a leaf next after going right three times and then left, when 
the vertex labeled with n, which is encoded by 1110, is visited. Finally, the last bit, O, 
leads to the leaf that is labeled with e. Therefore, the original word is sane. 
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We can construct a prefix code from any binary tree where the left edge at each 
interna] vertex is labeled by 0 and the right edge by a ! and where the leaves are labeled 
by characters. Characters are encoded with the bit string constructed using the labels 
of the edges in the unique path from the root to the leaves. 

There are algorithms, such as Huffman coding, that can be used to produce effi- 
W&f? eient codes based on the frequencies of occurrences of characters. We will not present 
the details of such algorithms here, (The interested reader can find the details of such 
algorithms in the references given for this section at the end of the book.) 


Exercises 

1. Build a binary search tree for the words banana, peach, 
apple, pear, coconut, mango, and papaya using alpha¬ 
betical order. 

2. Build a binary search tree for the words oenology, 
phrenology, campanology, ornithology, ichthyology, 
limnology, alchemy, and astrology using alphabetical 
order. 

3. How many comparisons are needed to locate or to 
add each of the following words in the search tree for 
Exercise 1, starting fresh each time? 

a) pear b) banana 

el kumquas d) orange 

4. riow many comparisons are needed to locate or to 
add each of the following words in the search tree for 
Exercise 2, starting fresh each time? 

a) palmistry 

b) etymology 

c) paleontology 

d) glaciology 

5. Using alphabetical order, construct a binary search tree 
for the words in the sentence, "The quick brown fox 
jumps over the lazy dog," 

6. How many weighings of a balance scale are needed to 
find a lighter counterfeit coin among four coins? De¬ 
scribe an algorithm to find the lighter coin using this 
number of weighings. 

7. How many weighings of a balance scale are needed to 
find a counterfeit coin among four coins if the coun¬ 
terfeit coin may be either heavier or lighter than the 
others? Describe an algorithm to find the counterfeit 
coin using this number of weighings, 

*8. How many weighings of a balance scale are needed to 
find a counterfeit coin among eight coins if the coun¬ 
terfeit coin is either heavier or lighter than the others? 
Describe an algorithm to find the counterfeit coin using 
this number of weighings. 

*9. How many weighings of a balance scale are needed to 
find a counterfeit coin among 12 coins if the counterfeit 
coin is lighter than the others'! 1 Describe an algorithm 
to find the lighter coin using this number of weighings. 


*10. One of four coins may be counterfeit, ff it is counter¬ 
feit, it may be lighter or heavier than the others. How 
many weighings are needed, using a balance scale, to 
determine whether there is a counterfeit coin, and if 
there is, whether it is lighter or heavier than the oth¬ 
ers? Describe an algorithm to find the counterfeit coin 
and determine whether it is lighter or heavier using this 
number of weighings. 

11, Which of the following codes are prefix codes:' 

a) 0 , 1 1, eOO, t: 10, ,r:01 

b) a:0, ft I, (:01,.r:OOI 

c) «:10l,r:ll, r:00l,.riOH.mOlO 

d) a:()IO, e:ll, t:01l, j: 101J, n; 1001, /:I0101 

12, Construct the binary tree with prefix codes representing 
the following coding schemes. 

a) mil, e:Q, t; 101 , 1.100 

b) a: 1, e;01, /:001, j: 0001, mOOOOl 

e) mlOlO. rf), fill, f:1011,*:IQ01,f:10001 

13, What are the codes for a, e, i, k, o, p, and «if the coding 
scheme is represented by the following tree ? 



14. Given the coding-scheme «:001, b:0001, <?:1, rrOOOO, 
j: 0100, f:011, r:0!0IO. find the Word represented by 

a) 01110100011. 

b) 0001110000. 
c OlOOlfliOIO. 
d) 01100101010. 
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8.3 _ 

Tree Traversal 


INTRODUCTION 


Ordered rooted trees are often used to store information. We need procedures for visiting 
Wvb each vertex of an ordered rooted tree to access data. We will describe several important 
algorithms for visiting ail the vertices of an ordered rooted tree. Ordered rooted trees can 
also be used to represent various types of expressions, such as arithmetic expressions 
involving numbers, variables, and operations. The different listings of tire vertices uf 
ordered rooted trees used to represent expressions arc useful in the evaluation of these 
expressions. 


UNIVERSAL ADDRESS SYSTEMS 

Procedures for traversing all vertices of an ordered rooted tree rely on the orderings of 
children. In ordered rooted trees, the children of an internal vertex are shown from left 
to right in the drawings representing these directed graphs. 

We will describe one way to order totally the vertices of an ordered rooted tree. 
To produce Lhis ordering, we must first label all the vertices. We do this recursively as 
follows. 

1. Label the root with die integer 0. Then label its k children (at level 1) from left to 

right with 1,2, .1. k , 

2, For each vertex v at level n with label A, label its k v children, as they are drawn 
from left to right, w ith A. 1, AX .... A,k v . 

Following this procedure, a vertex v at level n, for n > 1, is labeled jcj . x 2 . ■ ■ *. x n , where 
the unique path from the root to v goes through the xy th vertex at level 1, the xyth vertex 
at level 2, and so on. Tins labeling is called the universal address system of the ordered 
rooted tree. 

We can totally order the vertices using the lexicographic ordering of their labels in 
the universal address system. The vertex labeled x t .x 2 . ‘ • * is less than the vertex 

labeled yi-Vj. ■ -■ y m if there is an i, 0<i< n, with jr, = y T . x 2 = v 2 . jc,- i = 

>7-and x, < y,; or if u < m and j; - y, for i - 1. 2__ n. 


EXAMPLE 1 We display the labelings of the universal address system next to the vertices in the 

ordered rooted tree shown in Figure 1 , The lexicographic ordering of the labelings is 

0 < :I < 1.1 < 1.2 < 1.5 <2 <3 < 3.1 < 3.1.1 < 3.1,2 < 3.1.2.1 < 3,1.2.2 
< 3.1.2.3 < 3.1. 2.4 < 3.1.3 < 3.2 < 4 <4. 1 < 5 <5.1 <5.1.1 < 5.2 < 5.3 ■ 

TRAVERSAL ALGORITHMS 

Procedures for systematically visiting every vertex of an ordered rooted tree are called 
traversal algorithms. We will describe three of the most commonly used such algo¬ 
rithms, preorder traversal, inorder traversal, and postorder traversal. Each of these 
algorithms can be defined recursively. We lirst define preorder traversal. 
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_ 3.i.2.2 3.1.2.4 _ 

FIGURE 1 The Universal Address System of an Ordered Rooted Tree. 


DEFINITION1. Let f be an ordered rooted with root r. If 7 1 consists only of 
t, then r is die preorder traversal of T. Otherwise, suppose that T\, T 2 , . ■., T„ are 
the subtrees at r from left to right in T. The preorder traversal begins by visiting r. 
It continues by traversing T 1 in preorder, then Tj in preorder, and so on, until T n is 
traversed in preorder. 


The reader should verify that the preorder traversal of an ordered rooted tree gives the 
same ordering of the vertices as the ordering obtained using a universal address system. 
Figure 2 indicates how a preorder traversal is carried out. 



i n preorder in preorder in preorder 

FIGURE 2 Preorder Traversal. 
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EXAMPLE 2 



FIGURE 3 The Ordered Rooted Tree T, 


The following example illustrates preorder traversal. 


In which order does a preorder traversal visit the vertices in the ordered rooted tree T 
shown in Figure 3? 

Solution: The steps of the preorder traversal of T are shown in Figure 4. We traverse 
T in preorder by first listing the root a , followed by the preorder list of the subtree with 
root b the preorder list of the subtree with root c (which is just r) and the preorder list 
of the subtree with root d. 

The preorder list of the subtree with root b begins by listing b, then the vertices of 
the subtree with root e in preorder, and then the- subtree with root / in preorder (which 
is just/). The preorder list of the subtree with root d begins by listing d, followed by 
the preorder list of the subtree with rout g, followed by the subtree with root h (which 
is just h), followed by the subtree with root i (which is just i). 

The preorder list of the subtree with root e begins by listing e, followed by the 
preorder listing of the subtree with root j (which is just;), followed by the preorder 
listing of the subtree with root k. The preorder listing of the subtree with root g is 
g followed by l, followed by m. The preorder listing of the subtree with root k is k, 
n, o, p. Consequently, the preorder traversal of T is a, b, e, j, k, n, a, p,f c, d, g, l, m, 
h, i, W 


We will now define inorder traversal. 


DEFINITION 2. Let T be an ordered rooted tree wife root r. If T consists only 
of r then r is fee inorder traversal of T. Otherwise, suppose Chat T\, T%,..,T* arc 
the subtrees at r from left to right. The inorder traversal begins by traversing Ti in 
inorder, then visiting r. It continues by traversing T 2 in inorder, then T 3 in inorder, 
.... and finally T n in inorder. 


Figure 5 indicates how inorder traversal is carried out. 
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Preordsir traversal: Visit root, 
visit subtrees left to right 


a bed 




nop 



a b e j k f e d g i m h i 



nop 


abejknnpfrdgfmki 


FIGURE 4 The Preorder Traversal of T. 
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EXAMPLE 3 


EXAMPLE 4 



inorder inonlcr inorder 

FIGURE 5 Inorder Traversal, 


The following example illustrates how inorder traversal is carried out. 


In which order does an inorder traversal visit the vertices of the ordered rooted tree T 
in Figure 3? 

Solution: The steps of the inorder traversal of the ordered rooted tree T are shown in 
Figure 6. The murder traversal begins with an inorder traversal of the subtree with root 
b, the root a, the inorder listing of the subtree with root c. which is just c, and the inorder 
listing of the subtree with root d. 

The border listing of the subtree with root b begins with the inorder listing of the 
subtree with root e, the root b, and/. The inorder listing of the subtree with root d begins 
with the border iistittg of the subtree with root g, followed by the root d, followed by 
h, followed by i. 

The inorder listing of the subtree with root e is j, followed by the root e, followed 
by the inorder listing of the subtree with root k. The inorder listing of'the subtree with 
root >; is l, g, m. The inorder listing of the subtree with root k is n, k, o, p. Consequently, 
the inorder listing of the ordered rooted tree is j, e, n, k, o, p, b, f a, c, l, g, m, d, h, i. 


The definition of postorder traversal follows, 

DEFINITION 3. Let T be an ordered rooted tree with root r. If T consists only of 
r, then r is the postorder traversal of T. Otherwise, suppose that 7j, 7b,, l' n are 
the subtrees at r from left to right. The postorder traversal begins by traversing Ti 
in postorder, then 7'; in postorder.then T n b postorder, and ends by visiting r. 


Figure 7 illustrates how postorder traversal is done. The following example illustrates 
how postorder traversal works. 


In which order does a postorder traversal visit the vertices of the ordered rooted tree T 
shown in Figure 3? 
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Inurder tra versa!: Visii 

leftmost subtree, visit root, visit other 

subtrees left to right 



e b 



nop 



It 


jekbfdi Igmdhi 


nop 


jenkopbfac I g m d h i 


FIGURE 6 The Inorder Traversal of T. 


Solution: The steps of the postorder traversal of the ordered rooted tree T are shown in 
Figure 8. The postorder traversal begins with the postorder traversal of the subtree with 
root b, the postorder traversal of the subtree with root c, which is just c, the postorder 
traversal of the subtree with root d, followed by the root a. 

r fhe postorder traversal of the subtree with root b begins with the postorder traversal 
of the subtree with root e. followed by/, followed by the root b. The postorder traversal 
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in postorder in postorder in postorder 


FIGURE 7 Postorder Traversal. 


of the rooted tree with root d begins with the postorder traversal of the subtree with root 
g, followed by h, followed by /, followed by the root d. 

The postorder traversal of the subtree with root e begins with j, followed by the 
postorder traversal of the subtree with root k, followed by the root e. The postorder 
traversal of the subtree with root g is l, m, g. The postorder traversal of the subtree with 
root k is n, o, p, k. Therefore, the postorder traversal of T is j, n. o, p, k, e,f b, c, l, m, 
g, h, i, d, a. ■ 


There are easy ways to list the vertices of an ordered rooted tree in preorder, 
inorder, and postorder. To do this, first draw a curve around the ordered rooted tree 
starting at the root, moving along the edges as shown in the example in Figure 9. 
We can list the vertices in preorder by listing each vertex the first time this curve 
passes it. We can list the vertices in inorder by listing a leaf the first time the curve 
passes it and listing each internal vertex the second time the curve passes it. We 
can list the vertices in postorder by listing a vertex the last time it is passed on the 
way back up to its parent. When this is done in the rooted tree in Figure 9, it fol¬ 
lows that the preorder traversal gives a, b, d, h, e, i, j, c, f, g. k, the inorder traversal 
gives /i, d, b, i, e, j, a, f c, k. g, and the postorder traversal gives h, d, i, j, c, b, f 
k g, c, a. 

Algorithms for traversing ordered rooted trees in preorder, inorder, or postorder are 
most easily expressed recursively. 


ALGORITHM 1 Preorder Traversal. 

procedure preorder(T : ordered rooted tree) 
r : - root of T 
list r 

for each child c of r from left to right 
begin 

Tic ): = subtree with c as its root 
preorder(T{c)) 

end 
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Pustunle: Uin-criiil: Visit 
subtree, left to risht: visit root 


b c d 



f h r g k t d u 




tl a 


j ft <’ [> A t f h c I m V h i d a 


R(;LIRE 8 The Postorder Traversa! of 7\ 
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FIGURE 9 .4 Shortcut for Traversing an Ordered Rooted Tree in Proorder, Inorder, 
and Postorder, 


ALGORITHM 2 Inorder Traversal. 

procedure inorderiT: ordered rooted tree) 

r : = root of T 

if r is a leaf then list r 

else 

begin 

l : = first child of r from left to right 
T(l ); - subtree with / as its root 
inorderi Til)) 
list r 

for each child c of r except for l from left to right 
T (c): - subtree with c as its root 
inorder (Tic)) 

end 


algorithm .1 Postprdcr Traversal. 

procedure posiorderiT: ordered rooted tree) 
r : = root of 7’ 

for each child c of r from left to right 
begin 

T(r): = subtree with f as its root 
postorderiTic)) 
end 
list r 
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INFIX, PREFIX, AND POSTFIX NOTATION 


We can represent complicated expressions, such as compound propositions, combina¬ 
tions of sets, and arithmetic expressions using ordered rooted trees. For instance, con¬ 
sider the representation of an arithmetic expression involving the operators + (addi¬ 
tion). - (subtraction), * (multiplication), / (division), and f (exponentiation). We will 
use parentheses to indicate the order of the operations, An ordered rooted tret can be 
used to represent such expressions, where the internal vertices represent operations, and 
the leaves represent the variables or numbers. Each operation operates on its left and 
right subtrees (in that order). 


EXAMPLE 5 What is the ordered rooted tree that represents the expression ((a -f v) f 2) r 

<(.r - 4)/3)7 

Solution: The binary tree for this expression can be built from the bottom up. First, 
a subtree for the expression .v f v is constructed. Then this is incorporated as par! of 
the larger subtree representing (i + y) f 2. Also, a subtree for .r - 4 is constructed, 
and then this is incorporated into a subtree representing ( t -- 4)/3. Finally the subtrees 
representing (x + v) j 2 and (a - 4)/3 are combined to form the ordered rooted tree 
representing ((.i + y) \ 2) + «.v - 4)/3). These steps are shown in Figure 10. ■ 


An inorder traversal of the binary tree representing an expression produces the 
original expression with the elements mid operations in the same order as they orig¬ 
inally occurred, except for unary operations, which instead immediately follow their 
operands. For instance, infix traversals of the binary' trees in Figure 11, which represent 
the expressions (x + v)/( r + 3), (jc + :y/x>) -+■ 3, and x + {yi{x + 3)), all lead to the 
infix expression x + ylx -r 3. To make such expressions unambiguous it is necessary to 
include parentheses in the murder traversal whenever we encounter an operation. The 
fully parenthesized expression obtained in this way is said to be in infix form. 

We obtain the prefix form of an expression when we traverse its rooted tree in 
preorder, Expressions written in prefix form are said to be in Polish notation, which is 
named after the logician Jan Lukasiewicz (who was actually Ukrainian and not Polish). 
An expression in prefix notation (where each operation has a specified number of 



FIGURE 10 A Binary Tree Representing ox i yi t 2) t- fu - 4)/3i. 
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EXAMPLE 6 


EXAMPLE 7 


EXAMPLE 8 


web 



■' >• ? V r 

FIGURE 11 Rooted Trees Representing (x + v)/(r + 3), ix + ( yfx )) + 3, and x +■ {yf(x + 3)J. 


operands), is unambiguous, so no parentheses are needed in such an expression. The 
verification of this is left as an exercise for the reader. 


What is the prefix form for ({x + v) \ 2) + ((x - 4)/3)? 

Solution: We obtain the prefix form for this expression by traversing the binary tree 
that represents it. shown in Figure 10. This produces + f + x y 2 / - x 4 3. ■ 


In the prefix form of an expression, a binary operator, such as +, precedes its two 
operands. Hence, we can evaluate an expression in prefix form by working from right 
to left. When we encounter an operator, we perform the corresponding operation with 
the two operands immediately to the right of this operand. Also, whenever an operation 
is performed, we consider the result a new operand. 


What is the value of the prefix expression + - * 2 3 5 / * 2 3 4? 

Solution: The steps used to evaluate this expression by working right to left, and per- 
forming operations using the operands on the right, are shown in Figure 12. The value 
of this expression is 3. ■ 


We obtain the postfix form of an expression by traversing its binary tree in post- 
order. Expressions written in postfix form are said to be in reverse Polish notation. 
Expressions in reverse Polish notation are unambiguous, so parentheses are not needed 
The verification of this is left to the reader. 


What is the postfix fonn of the expression ((x + y) f 2) + ((x - 4)/3)? 


Jan Lukasiew icz 1 1878-1956). Jan Lukasiewicz, born in Lwow, studied and began his professional career 
at the University of Lwbw. He later moved to a professorship in Warsaw, After World War II, he -was ap¬ 
pointed ro a position ai the Royal Irish Academy in Dublin, Lukasiewicz worked in the area of many-valued 
logic; his 1921 paper on a three-valued logic w as an important contribution to this subject. Nevertheless, he 
is best known in the mathematical community far his introduction of parenthesis-free notation, now called 
Polish notation 
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EXAMPLE 9 


EXAMPLE 10 


+ - » 2 3 5 t t 2 3 4 

2 T 3 - 8 

*--*235/84 

L _i 

8/4=2 

+ -*2352 

L_,-1 

2*3 = ft 

+ -652 

6 5= 1 

+ l 2 
i_ j 

1+2 = 3 

Value of expression; 3 

FIGURE 12 Evaluating a Prefix Expression. 


Solution: The postfix form of the expression is obtained by carrying out a postorder 
traversal of the binary tree for this expression, shown in Figure 10. This produces the 
postfix expression: jt v + 2 \ x 4 - 3 / +. ■ 

In the postfix form of an expression, a binary operator follows its two operands. 
So, to evaluate an expression from its postfix form, work from left to right, carrying out 
operations whenever an operator follows two operands. After an operation is carried 
out, the result of this operation becomes a new operand. 


What is the value of the postfix expression 723*-4j93/+? 

Solution: The steps used to evaluate this expression by starting at the left and carrying 
out operations when two operands are followed by an operator are shown in Figure 13. 
The value of this expression is 4. ■ 


Rooted trees can be used to represent other types of expressions, such as those 
representing compound propositions and combinations of sets. In these examples unary 
operators, such as the negation of a proposition, occur. To represent such operators and 
their operands, a vertex representing the operator and a child of this vertex representing 
the operand are used. 


Find the ordered rooted tree representing the compound proposition ( -i(p A fl» ~ 
-uf). Then use this rooted tree to find the prefix, postfix, and infix forms of this 
expression. 
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7 2 7 * - 4 T 9 3 f + 

2*3=6 

7 6 - 4 T 9 3 1 + 

7-6=1 

1 4 7 9 3 / + 

l_ i 

1 4 = 1 


1 



9/3 = 3 


1 3 + 

I_j 

_ 1+3 = 4 _ 

FIGURE 13 Evaluating a Postfix Expression. 


Solution: The rooted tree for this compound proposition is constructed from the bottom 
up. First, subtrees for -i p and ->q are formed (where -i is considered a unary operator). 
Also, a subtree for p A q is formed. Then subtrees for -i (p A q) and (-i p) \/ (~iq) are 
constructed. Finally, these two subtrees are used to form the final rooted tree. The steps 
of this procedure are shown in Figure 14. 

The prefix, postfix, and infix forms of this expression are found by traversing this 
rooted tree in preorder, postorder, and rnorder (including parentheses), respectively. 
These traversals give -i A pq \j ->p-iq, pq A and (^i(p A q)) 

((-■P) V t -i <?)), respectively. ■ 


Because prefix and postfix expressions are unambiguous and because they can eas¬ 
ily be evaluated without scanning back and forth, they are used extensively in computer 
science. Such expressions are especially useful in the construction of compilers. 



FIGURE 14 Constructing the Rooted Tree for a Compound Proposition, 
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Exercises 


In Exercises 1-3 construct the universal address system for 
the given ordered rooted tree. Then use this to order its ver¬ 
tices using the lexicographic order of their labels. 


/!\ 

X 1 ' 

/ \ 

t \ 


A 


A 

7 \ 


\ 


/ \ 



4. Suppose that the address of the vertex v in the ordered 
rooted tree T is 3.4.5.2.+, 

a) At what level is r? 

b> What is the address of the parent of v’ 

c) Wltat is the least number of siblings v can have? 

d) What is the smallest possible number of vertices in 
T if v has this address? 

e) Find the other addresses that must occur. 

5. Suppose that the vertex with the largest address in an 
ordered rooted tree T has address 2,3.4,3. L Is it possi¬ 
ble to determine the number of vertices in T> 

6. Can die leaves of an ordered rooted tree have the fol¬ 
lowing list of universal addresses? If so. construct such 
an ordered rooted tree. 


a) 1.1.1. 1.1.2. 1.2, 2.1.1.I, 2.1.2, 2.1.3. 2.2, 3.1.1. 
3.1.2.1.3.1.2.2, 3,2 

b) 1.1, 1.2.1. 1.2.2, 1.2.3. 2.1. 2,2,1, 2.3.1, 2.3.2. 
2.4,2.1,2 4.2.2.3.1,3.2.1.3,2,2 

c) 1.1,1.2,1. 1.2.2,1,2.2.1,1.3,1.4,2,3.1, 3.2,4.1.1.1 


In Exercises 7-9 determine the order in which a preorder 
traversal visits the vertices of the given ordered rooted tree. 


7 . 


8. «a 



9 , « 




10, In which order are the vertices of the ordered rooted 
tree in Exercise 7 visited using an inorder traver¬ 
sal? 
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11. In which order are the vertices of the ordered rooted 
tree m Exercise <5 visited using an inorder traversal? 

12. In which order are the vertices of the ordered rooted 
tree in Exercise 9 visited using an inorder traversal? 

13. In which order are the vertices of the ordered rooted 
tree in Exercise 7 visited using a postorder traver¬ 
sal? 

14. In which order are the vertices of the ordered rooted 
tree in Exercise 8 visited using a postorder traver¬ 
sal'’ 

15. In which order are the vertices of the ordered rooted 
tree in Exercise 9 visited using a postorder traver¬ 
sal? 

16. Represent the expression {(.r + 2) * 3) • ( v — 1 3 * r ip 1 
using a binary tree. 

17. Write the expression in Exercise 16 in 

a) prefix notation. 

h) postfix, notation, 
c) mhx notation. 

18. Represent the expressions (• t _vy) + (x!y) and .v + 
{(t'y +• ai/yS using binary' trees. 

19. Write the expressions in Exercise 18 in 

a) prefix notation. 

b) postfix nutation, 
cj infix nutation. 

20. Represent the compound propositions -i(p.-\q) *-»■ 

( p\‘ -i q) and (-i p A (q -t p))\’ -'q using ordered 

rooted trees. 

21. Wriic the expressions in Exercise 20 in 
al prefix notation. 

hi postfix notation, 
tri infix notation- 

22. Represent [A n S) - {A U {B - A)} using an ordered 
rooted tree. 

23. Write the expression in Exercise 22 in 

a) prefix notation. 

b) postfix notation, 
e) infix notation. 

*24, In how nr any ways can the string : pA q —* ~i p \j - q 
be fully parenthesized to yield an infix expression? 

*25. In how many ways can the string A n fl - A n B - A 
be fully parenthesized to yield an infix expression? 

26, Draw the ordered rooted tree corresponding to each 
of the following arithmetic expressions written in pre¬ 
fix notation. Then write each expression using infix 
notation. 

al + * * - 53 2 1 4 

b) [ + 23-51 

e) - / 9 3 + * 2 4 ■ 7 6 

27. What is ihe value of each of the following prefix 
expressions 1 ’ 

ai •* 2 / 8 4 3 
hi ■ - * 3 3 * 4 2 5 

c) -+■ 132 123/6-42 

di * + 3 + 3 * 3 + 3 3 3 


28. What is the value of each of the following postfix 
expressions? 

a) 52 1 - -314 4-+* 

b) 93/5 + 72 - v 

c} 32*2 \ 53 - 84/*- 

29. Construct the ordered rooted tree whose preorder 
traversal is a, b. f, c, g, h, i, d, e, j, k, /, where a has 
four children, r has three children, j has two children, 
b and e have one child each, and all other vertices are 
leave*. 

*30. Show (hat an ordered rooted tree is uniquely deter¬ 
mined when a list of vertices generated by a preorder 
traversal of the tree and the number of children of each 
vertex are specified. 

*31. Show that an ordered rooted tree is uniquely deter¬ 
mined when a list of vertices generated by a postorder 
traversal of the tree and the number of children of each 
vertex are specified. 

32, Show that preorder traversals of the two ordered rooted 
trees displayed below produce the same list of ver¬ 
tices. Note that this does not contradict the state¬ 
ment in Exercise 30, since the numbers of children 
of internal vertices in the two ordered rooted trees 
differ. 


U 



a 



33, Show' that postorder traversals of the two ordered 
rooted trees (on the next page) produce the same list 
of vertices. Note that this does not contradict the state¬ 
ment in Exercise 31, since the numbers of children 
of internal vertices in the two ordered rooted trees 
differ. 
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Well-formed formulae in prefix notation over a set of sym¬ 
bols and a Set of binary operators are defined recursively by 
the following rules: 


f ij if x is a symbol, then i is a well-formed formula in 
prefix notation; 

(ii) if A' and Y are well-formed formulae and * is an oper¬ 
ator, then * XY is a well-formed formula. 

34. Which of the following are well-formed formulae over 
the symbols [x y, z} and the set of binary operators 

{x, -t- 7 o}? 

a) x +■ * x y x 

b) oxy Xxz 

c) x o x z X x x y 

d) X+oucrn 

*35. Show that any well-formed formula in prefix notation 
over a set of symbols and a set of binary operators 
contains exactly one more symbol than the number of 
operators. 

36. Give a definition of well-formed formulae in postfix 
notation over a set of symbols and a set of binary 
operators. 

37. Give six examples of well-formed formulae with three 
or more operators in postfix notation over the set of 
symbols [x, y. and the set of operators {+, x, o}, 

38. Extend the definition of well-formed formulae in pre¬ 
fix notation to sets of symbols and operators where the 
operators may not be binary. 


84 _ 

Trees and Sorting 

INTRODUCTION 


The problem of ordering the elements in a set occurs in many contexts. For instance, 
to produce a printed telephone directory it is necessary to alphabetize the names of the 
subscribers. 

Suppose that there is a total ordering of the elements of a set. Initially the elements 
in a set may be in any order, A sorting is a reordering of these elements into a list in 
which the elements are in increasing order. For instance, sorting the list 7,2, 1, 4,5, 9 
produces the list 1, 2, 4, 5, 7, 9. Sorting the list d, k, c, a ,/(using alphabetical order) 
produces the list a, c, d, f h. 

A large percentage of computer use is devoted to sorting one thing or another. 
Hence, much effort has been devoted to the development of efficient sorting algorithms. 
In this section several sorting algorithms and their computational complexity wifi be 
discussed, As will be seen in this section, trees are used to describe sorting algorithms 
and are used in the analysis of their complexity. 


THE COMPLEXITY OF SORTING 


Many different sorting algorithms have been developed. To decide whether a particular 
sorting algorithm is efficient, its complexity is determined. Using trees as models, a 
lower bound for the worst-case complexity of sorting algorithms can be found. 
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THEOREM 1 



KIGURE 1 A Decision Tree for Sorting Three Distinct Elements. 

There are n! possible orderings of n elements, since each of the a! permutations 
of these elements can be the correct order. The sorting algorithms we will study are 
based on binary' comparisons, that is, the comparison of two elements at a lime. The 
result of each such comparison narrows down the set of possible orderings. Thus, a 
sorting algorithm based on binary' comparisons can be represented by a binary decision 
tree in which each internal vertex represents a comparison of two elements, Each leaf 
represents one of the «! permutations of n elements. 

We display in Figure 1 a decision tree that orders the elements of the list a, b, c. ■ 


The complexity of a sort based on binary' comparisons is measured in terms of the 
number of such comparisons used. The most binary comparisons ever needed to sort 
a list with n elements gives the worst-ease performance of the algorithm. The most 
comparisons used equals the longest path length in the decision tree representing the 
sorting procedure. In other words, the most comparisons ever needed is equal to the 
height of the decision tree. Since the height of a binary tree with n\ leaves is at least 
[log n!j (using Corollary 1 in Section 8.1), at least [log «!] comparisons are needed, as 
stated in Theorem 1. 

A sorting algorithm based on binary comparisons requires at least [log nfj com¬ 
parisons. 

By Example 5 of Section 1.8, it follows that [log /:!] is 0\ n log n). In fact, greater 
than {n log n)l 4 for n > 4 (see Exercise 18), it fo I hows that no sorting algorithm that 
uses comparisons as the method of sorting can have worst-case time complexity that is 
better than 0(n log n) Consequently, a sorting algorithm is as efficient as possible (in 
the sense of a big-0 estimate of time complexity) if it has 0(n log n) time complexity. 

THE BUBBLE SORT 


The bubble sort is one of the simplest sorting algorithms but not one of the most efti- 
wef) cient. It puts a list into increasing order by successively comparing adjacent elements, 
interchanging them if they are in the wrong order. To carry out the bubble sort, we 
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perform the basic operation, that is, interchanging a larger element with a smaller one 
following it, starting at the beginning of the list, for a full pass. We iterate this procedure 
until the sort is complete. We can imagine the elements in the list placed in a column. 
In the bubble son, the smaller dements “bubble” to the top as they are interchanged 
with larger elements, The larger elements “sink"’ to the bottom. This is illustrated in the 
following example. 


Use the bubble sort to put 3, 2, 4, 1, 5 into increasing order. 

Solution: Begin by comparing the first two elements, 3 and 2. Since 3 > 2. interchange 
3 and 2, producing the list 2, 3, 4, 1, 5, Since 3 < 4, continue by comparing 4 and 1. 
Since 4 > 1, interchange ( and 4, producing the list 2, 3, 1,4, 5. Since 4 < 5, the first 
pass is complete. The first pass guarantees that the largest element, 5, is in the correct 
position. 

The second pass begins by comparing 2 and 3. Since these are in the correct order. 
3 and 1 are compared. Since 3 > 1, these numbers are interchanged, producing 2, 1,3, 
4, 5. Since 3 < 4, these numbers are in the correct order. It is not necessary' to do any 
more comparisons for this pass because 5 is already in the correct position. The second 
pass guarantees that the two largest elements. 4 and 5, are in their correct positions. 

The third pass begins by comparing 2 and I. These are interchanged since 2 > 1, 
producing I, 2, 3, 4, 5. Because 2 < 3, these two elements are in the correct order. It 
is not necessary to do any more comparisons for this pass because 4 and 5 are already 
in the correct positions. The third pass guarantees that the three largest elements. 3,4, 
and 5, are i n their correct positions. 

The fourth pass consists of one comparison, namely, the comparison of 1 and 2. 
Since 1 < 2. these elements are in the correct order. This completes the bubble sort. 
The steps of this algorithm are illustrated in Figure 2. ■ 


A pseudocode description of the bubble sort is given in Algorithm 1, 

How efficient is the bubble sort'.' Since n - i comparisons are used during the ith 
pass, the total number of comparisons used in a bubble sort of a list of n elements is 

Ui - 1.) + (n~ 2) + •■■ + 2 + 1. 

This is the sum of the n - 1 smallest integers. From Example 9 of Section 3,2, this 
equals (n - 1 )n/2. Consequently, the bubble sort uses n(n - 1 )/2 comparisons to order 
a list of n elements. (Note that the bubble sort always uses this many comparisons, 
since it continues even if the list becomes completely sorted at some intermediate step. ) 
Hence, the bubble sort algorithm has worst-case complexity 0(n 2 ). Since for every 


AL GORITHM 1 The Bubble Sort, 

procedure buhblesort(a h .... a„) 

for i : = I to n - 1 

begin 

for j \ = 1 to n - i 

if a, > then interchange a } and a;~\ 

end 

{u «„ is in increasing order} 
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FIG URE 2 The Steps of a Bubble Sort. 


positive real number c, n(n - l )/2 > cn log n for some sufficiently large positive integer 
n, it follows that the bubble sort does not have 0{n log n) worst-case time complexity. 
We need to find another algorithm in order to achieve this optimal estimate of worst-case 
compiexity. 


THE MERGE SORT 


Many different sorting algorithms achieve the best possible worst-case complexity for a 
weft sorting algorithm, namely, 0{n log n) comparisons to sort n elements. We will describe 
one of these algorithms, called the merge sort algorithm, here. We will demonstrate 
how the merge son algorithm works with an example before describing it in generality. 
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FIGURE 3 The Merge Sort of 8,2,4, 6, f >, % 1(1* 1,5,3. 


We will son the list 8, 2,4, 6.9, 7,10, 1,5,3 using the merge sort. A merge sort begins 
by splitting the list into individual elements by successively splitting lists in two, The 
progression of sublists for this example is represented with the balanced binary tree of 
height 4 shown in the upper half of Figure 3. 

Sorting is done by successively merging pairs of lists. At the first stage, pairs 
of individual elements are merged into lists of length two in increasing order. Then 
successive merges of pairs of lists are performed until the entire list is put into in¬ 
creasing order. The succession of merged lists in increasing order is represented by the 
balanced binary tree of height 4 shown in the lower half of Figure 3 (note that this tree 
is displayed "upside down”). ■ 


In general, a merge sort proceeds by iteratively splitting lists into two sublists of 
equal length (or where one sublist has one more element than the other) until each sub- 
list contains one element. This succession of sublists can be represented by a balanced 
binary tree. The procedure continues by successively merging pairs of lists, where both 
lists are in increasing order, into a larger list with elements in increasing order, un¬ 
til the original list is put into increasing order. The succession of merged lists can be 
represented by a balanced binary tree. 

We can also describe the merge sort recursively. To do a merge sort, we split a list 
into two sublists of equal, or approximately equal, size, sorting each sublist using the 
merge sort algorithm, and then merging the two lists. We leave it for the reader to give 
a complete specification of the recursive version of the merge sort. 
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TABLE 1 

Merging the Sorted Lists 2,3,5,6; and 1,4. 

First List 

Second List 

Merged List 

Comparison 

2 356 

1 4 


-- 

1 < 2 

2 356 

i 4 

1 

2 < 4 

356 

4 

1 2 

3 <4 

5 6 

5 6 

4 

i 

1 2 3 

12 34 

123456 

4 < 5 


An efficient algorithm for merging two ordered lists into a larger ordered list is 
needed to implement the merge sort. We will now describe such a procedure. 


VVe will describe how to merge the two lists 2, 3, 5, 6 and 1, 4. Table l illustrates the 
steps we use, 

First, compare the smallest elements in the two lists, 2 and 1, respectively. Since 
1 is the smaller, put it at the beginning of the merged list and remove it from the 
second list. At this stage, the first list is 2, 3, 5, 6, the second is 4, and the combined 
list is 1. 

Next, compare 2 and 4, the smallest elements of the two lists. Since 2 is the smaller, 
add it to the combined list and remove it from the first list. At this stage the first list is 
3,5, 6. the second is 4, and the combined list is 1,2. 

Continue by comparing 3 and 4, the smallest elements of their respective lists. 
Since 3 is the smaller of these two elements, add it to the combined list and remove it 
from the first list. At this stage the first list is 5, 6, and the second is 4. The combined 
list is 1. 2,3, 

Then compare 5 and 4, the smallest elements in the two lists. Since 4 is the smaller 
of these two elements, add it to the combined list and remove it from the second list. 
At this stage the first list is 5, 6, the second list is empty, and the combined list is 
1,2, 3, 4. 

Finally, since the second list is empty, all elements of the first list can be appended 
to the end of the combined list in the order they occur in the first list. This produces the 
ordered list 1, 2, 3,4, 5, 6. ■ 


We will now consider the general problem of merging two ordered lists L t and £ 2 . 
into an ordered list L We can use the following procedure. Start with an empty list L. 
Compare the smallest elements of the two lists. Put the smaller of these two elements 
at the end of L, and remove it from the list it was in. Next, if one of L\ and L, is empty, 
append the other (nonempty) list to L, which completes the merging. If neither L, nor 
Li is empty, repeat this process. Algorithm 2 gives a pseudocode description of this 
procedure. 

We will need estimates for the number of comparisons used to merge two ordered 
lists in the analysis of the merge sort. We can easily obtain such an estimate for Al¬ 
gorithm 2. Each time a comparison of an element from L\ and an element from Li is 
made, an additional element is added to the merged list L. However, when either L, 
or Li is empty, no more comparisons are needed. Hence, Algorithm 2 is least efficient 
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Ai GORJTHM : Merging Two Lists. 

procedure merge(L\. Lj : lists) 

L : ~ empty list 

while L\ arid L> are both nonempty 

begin 

remove smaller of first element of Li and Li from the list it is 
in and put it at the end of L 

if removal of this dement makes one list empty then remove 
all elements from the other list and append them to L 
cud {l is the merged list with elements in increasing order} 


when m + n — 2 comparisons are carried out, where m and n are the number of ele¬ 
ments in Li and L?, respectively, leaving one element in each of L, and L>. The next 
comparison w ill be the last one needed, because it will make one of these lists empty. 
Hence, Algorithm 2 uses no more than rn + n - l comparisons. The following lemma 
summarizes this estimate. 

Two sorted lists with m elements and n elements can be merged into a sorted list 
using no more than m + n - 1 comparisons. 

Sometimes two sorted lists of length m and n can be merged using far fewer than 
in ri - 1 comparisons. For instance, when m - 1, a binary search procedure can be 
applied to put the one element in the first list into the second list. This requires only 
flog n] comparisons, which is much smaller Shan m + n - 1 = n, for m =■ 1, On the 
other hand, for some values of m and«, Lemma ! gives the best possible bound. That is, 
there are lists with m and n elements that cannot be merged using fewer than m + n - i 
comparisons. (See Exercise 7 at the end of this section,) 

We can now analyze the complexity of the merge sort. Instead of studying the 
general problem, we will assume that n , the number of elements in the list, is a power 
of 2, say 2 m . This will make the analysis less complicated, but when this is not the case 
various modifications can be applied that will yield the same estimate. 

At the first stage of the splitting procedure, the list is split into two sublists, of 2 m 1 
elements each, at level I of the tree generated by the splitting. This process contin¬ 
ues, splitting the two sublists with 2 m_I elements into four sublists of 2 m " 2 elements 
each at level 2, and so on. In general, there are 2*“* lists at level k - 1, each with 
2 m *' 1 elements. These lists at level k -■ 1 are split into 2* lists at level k, each with 
2 m ~ k elements. At the end of this process, we have 2 m lists each with one element at 
level in. 

We start merging by combining pairs of the 2 m lists of one element into 2 m_! lists, 
at level m - 1, each with two elements. To do this, 2"' 1 pairs of lists with one element 
each are merged. The merger of each pair requires exactly one comparison. 

The procedure continues, so that at level k(k ~ m, m- 1 , m- 2,... , 3 , 2, 1 ), 2 * lists 
each with 2 M ' * elements are merged into 2*" 1 lists, each with 2 m “* +l elements, at level 
k- 1. To do this a total of 2 i_1 mergers of two lists, each with 2 m ~ k elements, are needed. 
But, by Lemma I, each of these mergers can be carried out using at most 2 m ~ k + 2 W_ k - 
I _ 2 m " i+1 - i comparisons. Hence, going from level k to k - I can be accomplished 
using at most 2 k ' I (2 m ^ t+1 - 1} comparisons. Summing all these estimates shows that 
the number of comparisons required for the merge sort is at most 
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\r 2 k , (2 ra ' t+1 - 1 ) = V?" Tr ! = m2'" - (2* “ I ) = n log b - n + l, 

it — L k =! i -! 

since m - log n and n - 2 m . (We evaluated V“ =] 2 m by noting that it is the sum of 
m identical terms, each equal to 2™. We evaluated X*= \ 2* 1 using the formula for the 
sum of the terms of a geometric progression from Example 6 of Section 3.2.) 

This analysis shows that the merge sort achieves the best possible big-O estimate 
for the number of comparisons needed by sorting algorithms, as stated in the following 
theorem. 

THEOREM 2 The number of comparisons needed to merge-sort a list with n elements is 0(n log n). 

We describe another efficient algorithm, the quick sort, in the exercises. 


Exercises 


t. Use a bubble sort to sort 3.1, 5, 7, 4, showing the lists 
obtained at each step. 

2. Use a bubble son to sort d, f, k. m a, b. showing the 
lists obtained at each step. 

*3. Adapt the bubble sort algorithm so that it slops when no 
interchanges are required, Express this more efficient 
version of the algorithm in pseudocode. 

4. Use a merge sort to sort 4, 3, 2. 5, 1, 8. 7. fi. Show all 
the steps used hv the algorithm. 

5. Use a merge sort to sort b, d, a.J, g. h, *, p, o, k. Show 
all the steps used by the algorithm. 

6. How many comparisons are required to merge the fol¬ 
lowing pairs of lists using Algorithm 27 

a'J 1,3, 5, 7, 9; 2,4. 6, 8,10 
hi I, 2,3, 4, 5:6. 7, 8,9. 10 
c) 1.3.6, 7,8:2, 3,4,9, It) 

7. Show that there are lists with rn dements and n de¬ 
ments such that they cannot be merged into one sorted 
list using Algorithm 2 with fewer than hi t rt -1 com¬ 
parisons. 

*8. What is tiie least number of comparisons needed to 
merge any two lists in increasing order into one list 
in increasing order when the number of dements in the 
two lists are 
a) 1.4? ht 2.4' 

c) 3, 4'.’ d! 4. 4' 5 

The selection sort begins hy finding the least element in 
the list This element is moved to the from. Then the least 
dement among the remaining elements is found and put 
into the second position. This procedure is repeated until 
the entire list has been sorted. 

9. Sort the following lists using the selection sort, 
al 3.5.4. 1.3 b) 5,4,3.2, l rt 1,2.3,4, 5 


10. Write the selection sort algorithm in pseudocode, 

11. How many comparisons are used to perform a selection 
sort of ri items? 

The quick sort is an efficient algorithm. To sort 
ii], a? .a,, this algorithm begins by taking the first ele¬ 

ment a, and forming two sublists, the first containing those 
dements that are less than ijj, in the order they arise, and 
the second containing those elements greater than «[, in the 
order they arise. Then U; is put at the end of the first sublist. 
This procedure is repeated recursively for each sublist, un¬ 
til all sublists contain one item. The ordered list of n items 
is obtained hy combining the sublists of one item in the 
order they occur. 

12. Sort 3, 5, 7. 8, 1. 9, 2. 4, 6 using the quick sort. 

13. Let O], U;, be a list of n distinct real numbers. 
How many comparisons are needed to form two sub- 
lists from this list, the first containing elements less 
than ci] and Lhe second containing elements greater 
than o\ 

14. Describe the quick sort algorithm using pseudocode. 

15. What is the largest number of comparisons needed to 
order a list of four elements using the quick sort algo¬ 
rithm? 

16. What is the least number of comparisons needed to or¬ 
der a list of four elements using the quick sort algo¬ 
rithm? 

17. Determine the worst-case complexity of the quick 
son algorithm in terms of the number of comparisons 
used. 

*18, Show that log ti is greater than (n log n)/4 for 
it > 4, [Hint: Begin with the inequality n’ > n(n - J) 
(ft - 2) ■ ■■ n i 2j. | 

*19. Write the merge sort algorithm in pseudocode. 
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Spanning Trees 


INTRODUCTION 

Consider the system of roads in Maine represented by the simple graph shown in 
Figure 1 (a). The only way the roads can be kept open in the winter is by frequently 
plowing them. The highway department wants to plow the fewest roads so that there 
will always be cleared roads connecting any tw r o towns. How can this be done? 

At least five roads must be plowed to ensure that there is a path between any 
two towns. Figure 1(b) shows one such set of roads, Note that the subgraph repre¬ 
senting these roads is a tree, since it is connected and contains six vertices and five- 
edges. 

This problem was solved with a connected subgraph with the minimum number 
of edges containing all vertices of the original simple graph. Such a graph must be a 
tree. 

DEFINITION 1. Let G be a simple graph. A spanning tree of G is a subgraph of 
G that is a tree containing every vertex of G, 



hill; 


Old Town 



FHjL KK 1 (a) A Road System and (b) A Set of Roads to Plow. 
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FIG i: RE 2 The Simple Graph C. 


A simple graph with a spanning tree must be connected, since there is a path in the 
spanning tree between any two vertices. The converse is also true; that is, every con¬ 
nected simple graph has a spanning tree. We will give an example before proving this 
result. 


Find a spanning tree of the simple graph G shown in Figure 2. 

Solution: The graph G is connected, but it is not a tree because it contains simple 
circuits. Remove the edge -[a, e}. This eliminates one simple circuit, and the resulting 
subgraph is still connected and still contains every vertex of G. Next remove the edge 
{e. /} to eliminate a second simple circuit. Finally, remove edge {c, to produce a 
simple graph with no simple circuits. This subgraph is a spanning tree, since it is a tree 
that contains every vertex of G. The sequence of edge removals used to produce the 
spanning tree is illustrated in Figure 3. 

The tree shown in Figure 3 is not the only spanning tree of G. For instance, each 
of the trees shown in Figure 4 is a spanning tree of G. ■ 


h c d a b v da b c d 



Htlie removed: (a. t; (?,/): (<-, el 

(a) (b) ft) 

FIG FRF 3 Producing a Spanning Tree for G by Removing Edges That Form Simple 
Circuits. 



FIGURE 4 Spanning Trees of C. 
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THEOREM 


EXAMPLE 


l A simple graph is connected if and only if it has a spanning tree. 

Proof: First, suppose that a simple graph G has a spanning tree T. T contains ever/ 
vertex of (/. Furthermore, there is a path in T between any two of its vertices. Since T 
is a subgraph of G, there is a path in G between any two of its vertices. Hence, G is 
connected. 

Now suppose that G is connected. If G is not a tree, it must contain a simple circuit. 
Remove an edge from one of these simple circuits. The resulting subgraph has one fewer 
edge but still contains all the vertices of G and is connected. If this subgraph is not a tree, 
it has a simple circuit; su as before, remove an edge that is in a simple circuit. Repeat 
this process until no simple circuits remain, This is possible because there are only a 
finite number of edges in the graph. The process terminates when no simple circuits 
remain. A tree is produced since the graph stays connected as edges are removed. This 
tree is a spanning tree since it contains every vertex of G. Q 

Spanning trees are important in data networking, as the following example shows. 


IF Multicasting Spanning trees play an important role in multicasting over Internet 
Protocol (IP) networks. To send data from a source computer to multiple receiving com- 
Wt'b puters, each of which is a subnetwork, data could be sent separately to each computer. 
This type of networking, called unicasting, is inefficient, since many copies of the same 
data are transmitted over the netw ork. To make the transmission of data to multiple re¬ 
ceiving computers more efficient, IP multicasting is used, With IP multicasting, a com¬ 
puter sends a single copy of data over the network, and as data reaches intermediate 
routers the data are forwarded to one or more other routers so that ultimately all receiv¬ 
ing computers in their various subnetworks receive these data. (Routers are computers 
that are dedicated to forwarding IP datagrams between subnetworks in a network. In 
multicasting, routers use Class D addresses, each representing a session that receiving 
computers may join; see Example 8 in Section 4.3.) 

For data to reach receiving computers as quickly as possible, there should be no 
loops (which in graph theory terminology are circuits or cycles) in the path that data 
take through the network. That is, once data have reached a particular router, data should 
never return to this router. To avoid loops, the multicast routers use network algorithms 
to construct a spanning tree in the graph that has the multicast source, the routers, and 
the subnetworks containing receiving computers as vertices, with edges representing 
the links between computers and/or routers. The root of this spanning tree is the mci- 
ticast source. The subnetworks containing receiving computers are leaves of the tree. 
(Note that subnetworks not containing receiving stations tire not included in the graph.) 
This is illustrated in Figure 5. ■ 


ALGORITHMS FOR CONSTRI CTING SPANNING TREES 

The proof of Theorem 1 gives an algorithm for finding spanning trees by removing 
web edges from simple circuits. This algorithm is inefficient, since it requires that simple 
circuits be identified. Instead of constructing spanning trees by removing edges, span¬ 
ning trees can be built up by successively adding edges. Two algorithms based cm this 
principle will be presented here. 

We can build a spanning tree for a connected simple graph using a depth-first 
search. We will form a rooted tree, and the spanning tree will be the underlying 
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FIGURE 5 A Multicase Spanning Tree, 


undirected graph of this rooted tree. Arbitrarily choose a vertex of the graph as the root. 
Form a path starting at this w x by successively adding edges, where each new edge 
is incident with the last vertex in the path and a vertex not already in the path. Continue 
adding edges to this path as long as possible. If the path goes through all vertices of the 
graph, the tree consisting of this path is a spanning tree. However, if the path does not 
go through all vertices, more edges must be added. Move hack to the next to last vertex 
in the path, and, if possible, form a new path starting at this vertex passing through 
vertices tbai were not already visited. If this cannot be done, move back another vertex 
in the path, that is, two vertices back in the path, and try again. Repeat this procedure, 
beginning at the last vertex visited, moving back up the path one vertex at a time, 
forming new paths that are as long as possible until no more edges can be added. Since 
the graph has a finite number of edges and is connected, this process ends with the 
production of a spanning tree. Each vertex that ends a path at a stage of the algorithm 
will be a leaf in the rooted tree, and each vertex where a path is constructed starting 
at this vertex will be an internal vertex. The reader should note the recursive nature of 
this procedure, Also, note that if the vertices in the graph are ordered, the choices of 
edges at each .stage oi the procedure are all determined when we always choose the first 
vertex in the ordering that is available. However, we will not always explicitly order 
the vertices of a graph. 

Depth-first search is also called backtracking, since the algorithm returns to ver¬ 
tices previously visited to add paths. The following example illustrates backtracking. 


Use a depth-first search to find a spanning tree for the graph G shown in Figure 6. 

Solution: The steps used by a depth-first search to produce a spanning tree of G are 
shown in Figure 7. We arbitrarily start with the vertex/ A path is built by successively 
adding edges incident with vertices not already in the path, as long as this is possible. 
T his produces a path f g, k, k, j (note that other paths could have been built). Next, 
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FIGURE 6 The Graph G. 


backtrack to k. There is nu path beginning at k containing vertices not already visited. 
So we backtrack to h. Form the path h, i. Then backtrack to h, and then to/ From/ 
build the path/ d. e, c, a. Then backtrack to c and form the path c, b. This produces the 
spanning tree, ■ 


We can also produce a spanning tree of a simple graph by the use of a breadth-first 
search. Again, a rooted tree will be constructed, and the underlying undirected graph 
of this rooted tree forms the spanning tree. Arbitrarily choose a root from the vertices 
of the graph. Then add all edges incident to this vertex. The new vertices added at 
this stage become the vertices at level 1 in the spanning tree. Arbitrarily order them. 
Next, for each vertex at level 1, visited in order, add each edge incident to this vertex 
to the tree as long as it does not produce a simple circuit. Arbitrarily order the children 
of each vertex at level 1. This produces the vertices at level 2 in the tree. Follow the 
same procedure until all the vertices in the tree have been added. The procedure ends 
since there are only a finite number of edges in the graph. A spanning tree is produced 
since we have produced a tree containing every 1 vertex of the graph. An example of a 
breadth-first search foilotvs. 


Use a breadth-first search to find a spanning tree for the graph shown in Figure 8. 

Solution: The steps of the breadth-first search procedure are shown in Figure 9. We 
choose the vertex e to be the root. Then we add edges incident with all vertices adjacent 
to e, so that edges from e to b. d,f, and i are added. These tour vertices are at level 1 
in the tree. Next, add the edges from these vertices at level 1 to adjacent vertices not 
already in the tree. Hence, the edges from h to a and c are added, as tire edges from d 



FIGURE 7 A Depth-First Search of G, 
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EXAMPLE 5 



m k 


FIGURES A Graph G. 


to h, from/ to j and g, and from i to t The new vertices a , c, h,j, g, and k are at level 2. 
Next, add edges from these vertices to adjacent vertices not already in the graph. This 
adds edges from g to / and from k to m, ■ 

BACKTRACKING 


There are problems that can be solved only by performing an exhaustive search of all 
possible solutions. One way to search systematically fora solution is to use a decision 
tree, where each internal vertex represents a decision and each leaf a possible solution. 
To find a solution via backtracking, first make a sequence of decisions in an attempt 
to reach a solution as long as this is possible. The sequence of decisions can be repre¬ 
sented by a path in the decision tree. Once it is known that no solution can result from 
any further sequence of decisions, backtrack to the parent of the current vertex and 
work toward a solution with another series of decisions, if this is possible. The proce¬ 
dure continues until a solution is found, or it is established that no solution exists. The 
following examples illustrate the usefulness of backtracking. 


Graph Colorings How can backtracking be used to decide whether a graph can be 
colored using n colors? 

Solution: We can solve this problem using backtracking in the following way, First 
pick some vertex a and assign it color 1. Then pick a second vertex b, and if b is not 
adjacent to a, assign it color 1. Otherwise, assign color 2 to b. Then go on to a third 
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vertex c. Use color 1, if possible, fore. Otherwise use color 2, if this is possible. Only 
if neither color 1 nor color 2 can be used should color 3 be used. Continue this process 
as long as it is possible to assign one of the n colors to each additional vertex, always 
using the first allowable color in the list. If a vertex is reached that cannot be colored 
by any of the n colors, backtrack to the last assignment made and change the coloring 
of the last vertex colored, if possible, using the next allowable color in the list. If it 
is not possible to change this coloring, backtrack further to previous assignments, one 
step back at a time, until is is possible to change a coloring of a vertex. Then continue 
assigning colors of additional vertices as long as possible. If a coloring using n colors 
exists, backtracking will produce it, {Unfortunately this procedure can be extremely 
inefficient.) 

In particular, consider the problem of coloring the graph shown m Figure 10 with 
three colors. The tree shown in Figure ID illustrates how backtracking can be used to 
construct a 3-coloring, In this procedure, red is used first, then blue, and finally green. 
This simple example can obviously be done without backtracking, but it is a good il¬ 
lustration of the technique. 

In this tree, the initial path from the root, which represents the assignment of red 
to a, leads to a coloring with a red, b blue, c red, and d green. It is impossible to color e 
using any of the three colors when a, b, c, and d are colored in this way. So, backtrack to 
the parent of the vertex representing this coloring. Since no other color can be used for 
cl backtrack one more level. Then change the color of c to green. We obtain a coloring 
of the graph by then assigning red to d and green to e, ■ 


The n-Quecns Problem The /i-queens problem asks how n queens can be placed on 
an n X n chessboard so that no two queens can attack one another. How can backtracking 
wtb be used to solve the n -queens problem? 


ti red 


a n.'d 
b Wee 



a rt*J 
b blue 
r jKen 
d led 

__ firCM 

FIGURE 10 Coloring a Graph Using Backtracking. 
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Sahelian: To solve this problem we must find n positions on an n Xn chess board so that 
no two of these positions are in the same row. same col umn, or in the same diagonal 
[a diagonal consists of all positions (i, j) with i + j = m for some m, or i - j - m 
for some m|. We will use backtracking to solve the n-queens problem. We start with 
an empty chessboard. At stage k + 1 we attempt putting an additional queen on the 
board in the (k + l)st column, where there are already queens in the first, k columns. 
We examine squares m I he (k + list column starting with the square in the first row, 
looking for a position to place this queen so that is not in the same row or on the same 
diagonal as a queen already on the board. (We already know it is not in the same col¬ 
umn.) If it is impossible to find a position to place the queen in the (k + l)st column, 
backtrack to the placement of the queen in the kxh column, and place this queen in the 
next allowable row in this column, if such a row exists. If no such row exists, backtrack 
further. 

In particular, Figure 11 displays a backtracking solution to the four-queens problem. 
In this solution, we place a queen in the first row and column. Then we put a queen in 
the third row of the second column. However, this makes it impossible to place a queen 
in the third column. So we backtrack and put a queen in the fourth row of the second 
column. When we do this, we can place a queen in the second row of the third column. 
But there is no way to add a queen to the fourth column. 'l*his shows that no solution 
results when a queen is placed in the first row and column. We backtrack to the empty 
chessboard, and place a queen in the second row of the first column. This leads to a 
solution as shown in Figure II. ■ 




X represents a queen 

FIGURE 11 A Backtracking Solution of the Four-Queens Problem, 
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0 

Sum - U 



{31,7} 131.5} |27,11) (27,7) 

Stun = 38 Sum = 36 Sum = 38 Sum = 34 



[27, 7,5) 

Sum -■ 39 

FIGURE 12 Find a Sum Equal fo 39 Using Backtracking, 


EXAMPLE? Sums of Subsets Consider the following problem. Given a set of positive integers 

t], X 2 .c„, find a subset of this set of integers that has M as its sum. How can back¬ 

tracking be used to solve this problem? 

Solution: We start with a sum with nn terms. We build up the sum by successively 
adding terms. An integer in the sequence is included if the sum remains less than M 
when this integer is added to the sum. If a sum is reached so that the addition of any 
term is greater than M, backtrack by dropping the last term of the sum. 

Figure 12 displays a backtracking solution to the problem of finding a subset of 
{31,27,15, 11,7,5} with sum equal to 39. ■ 


Exercises 


1. How many edges must be removed from a connected 
graph with n vertices and m edges to produce a span¬ 
ning tree'.’ 


Ill Exercises 2-6 find a spanning tree for the graph shown 
by removing edges in simple circuits. 


2 . 





3. 




K / f 


a h 
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7. I ■ imi a spanning tree tor each of the following graphs, 
a) hi A ' 4 .4 c) 

dl Q;, el r, f) W 5 

In Exercises 8-10 draw all the spanning trees of the given 
simple graphs. 


8 . 



d r h 


30. 



d 


*11. How many different spanning trees does each of the 
following simple graphs have? 
a) K< b) A', c) A.^ d) C 5 

*12. How many nomsoniorphic spanning trees does each of 
the following simple graphs have? 
a) A. b) A. c) A\ 


in Exercises 13-15 use a depth-first search to produce a 
spanning tree for the given simple graph. Chouse a as the 
root of this spanning tree and assume that the vertices are 
ordered alphabetically. 



h f g / 



16. Use a breadth-first search to produce a spanning tree for 
each of the simple graphs in Exercises 13-15. Choose 
a as the root of each spanning tree, 

17. Suppose that an airline must reduce its flight schedule 
to save money. If its original routes are as illustrated in 
the figure at the bottom of the page, which flights can 
be discontinued to retain service between all pairs of 
cities (where it may be necessary to combine (lights to 
fly from one city to another)? 

18. When must an edge of a connected simple graph be in 
every spanning tree for this graph? 

19. Which connected simple graphs have exactly one s pan¬ 
ning tree? 


Harbor 
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20. Explain how a breadth-first search or a depth-first 
search can he used to order the vertices at a connected 
graph 

*21. Write Ihe depth-first search procedure in pseudocode. 
*22. Write the breadth-first search procedure in pseudocode. 
*23. Show thai the length of the shortest path between ver¬ 
tices v and u in a connected simple graph equals the 
level number of u in the breadth-first spanning tree of 
(i with root v. 

24. Lise backtracking to try to find a coloring of each of 
the graphs in Exercises 5-7 of Section 7.8 using three 
colors. 

25. Use backtracking to solve the n-queens prob lem for the 
following values of n 

a) n - 3 b) n ~ 5 e) n = 6 

26. Use backtracking to find a subset, if ii exists, of the set 
{ 27 . 24 . iy 14, 11,8} with sum 

a) 20. b) 41. c) 60, 

27. Explain how backtracking can be used to find a Hamil¬ 
ton path or circuit in a graph, 

28. a) Explain how backtracking can be used to find the 

way out of amaze, given a starting position and the 
exit position. Consider Ihe maze divided into po¬ 
sitions, where at each position the set of available 
moves includes one to four possibilities (up, down, 
right, left). 

h) Find a path from the starting position marked by X 
to the exit in the following maze. 

rrrnwTi 


i 

fix II 

A spanning forest of graph G is a forest that contains every 
vertex ot G such that two vertices are in the same tree of the 
forest when there is apath in G between these two vertices, 

29. Show that every finite simple graph has a spanning 
forest. 

30. How many trees are in the spanning forest of a graph? 

31. How many edges must be removed to produce the span¬ 


ning forest of a graph with n vertices, m edges, and t 
connected components? 

32. Devise an algorithm for constructing the spanning for¬ 
est of a graph based on deleting edges that form simple 
circuits, 

33. Devise an algorithm for constructing the spanning for¬ 
est of a graph based on depth-first searching. 

34. Devise an algorithm for constructing the spanning for¬ 
est of a graph based on breadth-first searching. 

Let T\ and 7j be spanning trees of a graph, The distance 

between and 77 is the number of edges in 7 : and 7? that 

are not common to 7, and T 2 . 

35. Find the distance between each pair of spanning trees 
shown in Figures 3(c) and 4 of the graph G shown in 
Figure 2. 

*36. Suppose that 7}, 77. and 7 : , are spanning trees of the 
simple graph G. Show that the distance between T\ and 
T } does not exceed the sum of tire distance between T i 
and 7? and the distance between 7? and 1\. 

**37. Suppose that 7] and Ti are spanning trees of a simple 
graph C. Moreover, suppose that is an edge in 7| 
that is not in T>. Show that there is an edge e 2 in 7,. 
that is not in 7j such that 7j remains a spanning tree 
if fi is removed from it and e% is added to it. and T : 
remains a spanning tree 1 1 <?i is removed from it and ty 
is added to it. 

*38. Show that it is possible to find a sequence of spanning 
trees leading from any spanning tree to any other by 
successively removing one edge and adding 
another. 

A rooted spanning tree of a directed graph is a rooted tree 
containing edges of the graph such that every vertex of the 
graph is an endpoint of one of the edges in the tree. 

39. For each of the directed graphs in Exercises 24-28 of 
See Lion 7,5 either find a rooted spanning tree of the 
graph or determine that no such tree exists. 

*40. Show that a connected directed graph in which each 
vertex has the same m-degree and out-degree has a 
rooted spanning tree. (Hint: Use an Euler circuit.) 

*41, Give an algorithm to build a rooted spanning tree for 
connected directed graphs in which each vertex has the 
same in-degree and out-degree. 


8i>_ 

Minimum Spanning Trees 

INTRODUCTION 

W’f'/; a company plans to build a communications network connecting its ti ve computer cen¬ 
ters. Any pair of these centers can be linked w ith a leased telephone line. Which links 
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$2000 



FIGURE 1 A Weighted Graph Showing Monthly Lease Costs for Lines in a Computer 
Network, 


should be made to ensure that there is a path between any two computer centers so 
that the total cost of the network is minimized? We can model this problem using the 
weighted graph shown in Figure 1, where vertices represent computer centers, edges 
represent possible leased lines, and the weights on edges are the monthly lease rates of 
the lines represented by the edges. We can solve this problem by finding a spanning tree 
so that the sum of the weights of the edges of the tree is minimized. Such a spanning 
tree is called a minimum spanning tree. 


ALGORITHMS FOR MINIMUM SPANNING TREES 


A wide variety of problems are solved by finding a spanning tree in a weighted graph 
such that the sum of the weights of the edges in the tree is a minimum. 

DEFINITION X. A minimum spanning me in a connected weighted graph is a 
spanning tree that has the smallest possible sum of weights of its edges. 


We will present two algorithms for constructing minimum spanning trees. Both proceed 
by successively adding edges of smallest weight from those edges with a specified 
property' that have not already been used. These algorithms are examples of greedy 
algorithms. A greedy algorithm is a procedure that makes art optimal choice at each of 
its steps. Optimizing at each stage of an algorithm does not guarantee that the optimal 
overall solution is produced. However, the two algorithms presented in this section for 
constructing minimum spanning trees are greedy algorithms that do produce optimal 
solutions. 

The first algorithm that we will discuss was given by Robert Prim in 1957, although 
W€b the basic ideas of this algorithm have an earlier origin. To carry out Prim’s algorithm, 
begin by choosing any edge with smallest weight, putting it into the spanning tree. 
Successively add to the tree edges of minimum weight that are incident to a vertex 
already in the tree and not forming a simple circuit with those edges already in the tree. 
Stop when n - 3 edges have been added. 


Robert Clay Prim (born 1921). Robert Prim, bom in Sweetwater, Texas, received his B.S. in Electrical 
Engineering in 1941 and hisPh.D in Mathematics from Princeton University in 1949. He was an engineer 
at the General Electric Company from 1941 until 1944, an engineer and mathematician at the United States 
Naval Ordnance Lab from 1944 until 1949, and a research associate at Princeton University from 194S until 
1949. Among the other positions he has held are director of mathematics and mechanics research at Bell 
Telephone Laboratories from 1958 until 1961 and vice president of research at Sandia Corporation, He is 
currently retired. 
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EXAMPLE 1 


Later m this section, we will prove (hat this algorithm produces a minimum span¬ 
ning tree for any connected weighted graph. Algorithm 1 gives a pseudocode descrip¬ 
tion of Prim’s algorithm. 


ALGORITHM 1 Prim’s Algorithm. 

procedure Prim(G: weighted connected undirected graph with n 
vertices) 

T : - a minimum-weight edge 
for i 1 to n - "2 

1 begin 

e : = an edge of minimum weight incident to a vertex in T nod not 
forming a simple circuit in T if added to T 
T : = T with e added 

j end \ T is a minimum spanning tree of G] 


Note that the choice of an edge to add at a stage of the algorithm is not determined 
when there is more than one edge with the same weight that satisfies the appropriate 
criteria. We need to order the edges to make the choices deterministic. We will not 
worry about this in the remainder of the section. Also note that there may be more than 
one minimum spanning tree for a given connected weighted simple graph. (See Exer¬ 
cise 9.) The following examples illustrate how Prim’s algorithm is used. 



Use Prim’s algorithm to design a minimum-cosi communications network connecting 
all the computers represented by the graph in Figure 1, 

Solution: We solve this problem by finding a minimum spanning tree in the graph 
in Figure 1. Prim's algorithm is carried out by choosing an initial edge of minimum 
weight and successively adding edges of minimum weight that arc incident to a vertex 
in the tree and that do not form simple circuits. The edges in color irt Figure 2 show a 
minimum spanning tree produced by Prim's algorithm, with the choice made at each 
step displayed. ■ 



r,ta- 

Cost 

( Chicago. Attala. ] 

£ 700 

t Atlanta, New- York J 

t 800 

1 Chicago. San Francisco j 

$t2flft 

I San Francisco, Denver} 

S 900 

Total ■ 

S3600 


FIGURE 2 A Minimum Spanning Tree for the Weighted Graph in Figure 1. 
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EX AM PL F. 


b 3 r 1 4 


I- 
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— T 

1 

4 / 

j * 

3 I 

2 

4 

T 

i 1 


i i 


FIGURE 3 A Weighted G 


Use Prim’s algorithm to find a minimum spanning tree in the graph shown in Pigure 3. 

Solution: A minimum spanning tree constructed using Prim’s algorithm is shown in 
Figure 4. The successive edges chosen are displayed. ■ 


The second algorithm we will discuss was discovered by Joseph Kruskal in 1956, 
Web although the basic ideas it uses were described much earlier. To carry out Kruskal's 
algorithm, choose an edge in the graph with minimum weight. 


Ijtgc 



4 

5 

6 


ibn 
(«. b] 
!/.;} 


[«, ej 5 

Uj'l 1 

•1/: #1 J 

ir, jd 2 

k. <6 i 

i*. « i 

I ft, i I 3 

11. /1 j_ 


J 014*1: 24 


(b) 


FIGURE 4 A Minimum Spanning Tree Produced Using Prim’s Algorithm. 


Joseph Bernard Kruskal litflrn 1928). Joseph Kruskal, born in New York City, attended the University 
of Chicago tind received his Pli.D, from Princeton University m 1954. lie was an instruct tui in nmlliematks 
at Princeton and at the University of Wisconsin, and later he was an assistant professoral the University of 
Michigan. In 1959 he became a member of the technical staff at Bell Laboratories, a position ho continues 
to hold. His current research interests include statistical linguistics and psychometrics. Besides his work on 
minimum spanning irees, Kruskal is also known for contributions to multidimensional scaling. Kruskal dis¬ 
covered his algorithm for producing minimum spanning trees when be was a second-year graduate student. 
He was not sure his 1 16-page paper (in this subject was worthy of publication but was convinced by others 
to submit it. 

Historical Note: Joseph Kruskal and Robert Pnm developed their algorithms for constructing minimum 
spanning trees ill the mid-1950s. However, they were not the first people to discover such algorithms. T; or 
example, the work of the anthropologist Jan O.ekanuwski, in 1909, contains many of the ideas requited 
to lind minimum spanning frees. In 1926. Otakar Boruvka described methods for constructing minimum 
spanning trees in work relating to the. construction of electric power networks. 
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EXAMPLE 3 


Successively add edges with minimum weight that do not form a simple circuit 
with those edges already chosen. Stop after n — 1 edges have been selected. 

The proof that Kruskal’s algorithm produces a minimum spanning tree for every 
connected weighted graph is left as an exercise at the end of this section. Pseudocode 
for Kruskal's algorithm is given in Algorithm 2. 


VUiORITHH 2 Kruskal's Algorithm. 


1 


procedure Kruskat(G: weighted connected undirected graph with n 
vertices) 

T : - empty graph 
! for i: = 1 to n - 1 
begin 

e : = any edge in G with smallest weight that does not form a 
simple circuit when added to T 
7 :=- T with e added 
end {T is a minimum spanning tree of Gj 


The reader should note the difference between Prim’s and Kruskal’s algorithms. In 
Prim’s algorithm edges of minimum weight that are incident to a vertex already in the 
tree, and not forming a circuit, are chosen; whereas, in Kruskal’s algorithm edges of 
minimum weight that are not necessarily incident to a vertex already in the tree, and 
that do not form a circuit, are chosen. Note that as in Prim’s algorithm, if the edges 
are not ordered, there may be more than one choice for the edge to add at a stage of 
this procedure. Consequently, the edges need to be ordered for the procedure to be 
deterministic. The following example illustrates how Kruskal’s algorithm is used. 


Use Kruskal’s algorithm to find a minimum spanning tree in the weighted graph shown 
in Figure 3. 

Solution: A minimum spanning tree and the choices of edges at each stage of Kruskal’s 
algorithm are shown in Figure 5. ■ 


u 2 b S <• 1 d 
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FIGl'RE 5 A Minimum Spanning Tree Produced by Kruskal’s Algorithm. 
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We will now prove that Prim’s algorithm produces a minimum spanning tree of a 
connected weighted graph. 

Proof: Let G be a connected weighted graph. Suppose that the successive edges cho¬ 
sen by Prim’s algorithm are fj, e% .. <?„_j ■ Lei *5 be the tree with e<, r 2 ,.. . o, . as 

its edges, and let S l: be the tree with <?i, e 2 .as its edges. Let T be a minimum 

spanning tree of G containing the edges e\, e 2 ,,.., where k is the maximum inte¬ 
ger with the property (hat a minimum spanning tree exists containing the first k edges 
chosen by Prim’s algorithm. The theorem follows if we can show that S - T. 

Suppose that S ¥= T, so that k < n - 1. Consequently, T contains <?i, e 2 . e k , 

but not ejj + |. Consider the graph made up of T together with Since this graph 
is connected and has n edges, loo many edges to be a tree, it must contain a simple 
circuit. This simple circuit must contain e*, i since there w as no simple circuit in T. 
Furthermore, there must be an edge in the simple circuit that does not belong to Sj,_ [ 
since S*+i is a tree. By starting at an endpoint of <?* +1 that is also an endpoint of one 

of the edges e\ .e*. and following the circuit until it reaches an edge not in 

we can find an edge e not in S* +1 that has an endpoint that is also an endpoint of one 
of the edges ej, e 2 , ■ ■ ■, e k . By deleting e from T and adding e k + 1 , we obtain a tree 
7” with n — 1 edges (it is a tree since it has no simple circuits}. Note that the tree T' 

contains e\, e 2 . _ e k , Furthermore, since was chosen by Prim’s algorithm 

at the (k + 1 }st step, and e was also available at that step, the weight of e k+ \ is less than 
or equal to the weight of c. From this observation it follows that T' is also a minimum 
spanning tree, since the sum of the weights of its edges does not exceed the sum of 
the weights of the edges of 71 This contradicts the choice of k as the maximum integer 
so that a minimum spanning tree exists containing rq, ... f e k . Hence, k - n - I , and 
S = T, It follows that Prim’s algorithm produces a minimum spanning tree. □ 


Exercises 


1. The roads represented by the following graph are all 
unpaved. The lengths of the roads between pairs of 
towns are represented by edge weights. Which roads 
should be paved, so that there is a path of paved roads 


between each pair of towns so that a minimum road 
length is paved? [Note: These towns are in Nevada.) 

In Exercises 2—4 use Print’s algorithm to find a minimum 
spanning tree for the given weighted graph. 
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4 . 



S. Use Kmskal s algorithm to design the communications 
network described at the beginning of the section. 

ft. Use Kruskai’s algorithm to find a minimum spanning 
tree for the weighted graph in Exercise 2. 

7. Use Kruskal's algorithm to find a minimum spanning 
tree for the weighted graph in Exercise 3. 

8, Use Kruskal's algorithm to find a minimum spanning 
Lree Tor the weighted graph in Exercise 4. 

4. Find a connected weighted simple graph with the 
fewest edges possible that has more than one minimum 
spanning tree. 

10. A minimum spanning forest in a weighted graph is 
a spanning iorcsi with minimal weight Explain how 
Print’s and Km skid's algorithms can he adapted to con¬ 
struct minimum spanning forests. 

A maximum spanning tree of a connected weighted undi¬ 
rected graph is a spanning tree with the largest possible 

weight. 

11. l)e vise an algorithm similar to Prim’s algorithm for 
constructing a maximum spanning tree of a connected 
weighted graph. 

12. Devise an algorithm similar to Kruskai’s algorithm for 
constructing a maximum spanning tree of a connected 
weighted graph. 

13. Kind a maximum spanning tree for the weighted graph 
in Exercise 2, 

14. Find a maximum spanning tree for the weighted graph 
in Exercise 3. 

15. Find a maximum spanning tree for the weighted graph 
in Exercise 4. 

16. Find (he second least expensive communications net¬ 
work connecting the live computer centers in the prob¬ 
lem posed at the beginning of the section, 

4 17. lie-vise an algorithm for finding the second shortest 
spanning tree in a connected weighted graph. 

51 18. .Show that an edge with smallest weight in a connected 
weighted graph must he part of any minimum spanning 
tree. 

19. Show that there is a unique minimum spanning tree in 
a connect ed weigh toil graph if the weights of the edges 
ate all ditl'etenl. 


20. Suppose that the computer network connecting the 
cities in Figure 1 must contain a direct link between 
New- York and Denver. What other links should be in¬ 
cluded so that there is a link between every two com¬ 
puter centers and the cost is minirniml? 

21. Find a spanning tree with minimal total weight contain¬ 
ing the edges {e, t} and {g, k} in Lhe weighted graph in 
Figure 3. 

22. Describe an algorithm for finding a spanning tree with 
minimal weight containing a specified set of edges in a 
connected weighted undirected simple graph 

23. Express the algorithm devised in Exercise 22 in pseu¬ 
docode. 

Sollin’s algorithm produces a minimum spanning tree 
from a connected weighted simple graph G = (U£j by 
successively adding groups of edges. Suppose that the ver¬ 
tices in V are ordered. This produces an ordering of the 
edges where {«o. t’ol precedes {«i. r ; } if un precedes m; or 
if uo - mj and i-vj precedes vj. The algorithm begins by si¬ 
multaneously choosing the edge of least weight incident to 
each vertex. The first edge in the ordering is taken in the 
case of ties This produces a graph with no simple circuits, 
that is. a forest of trees (Exercise 24 asks for a proo f of this 
fact). Next, simultaneously choose for each tree in the for¬ 
est the shortest edge between a vertex in this tree and a ver¬ 
tex in a different tree. Again the first edge in the ordering 
is chosen in the ease of ties. (This produces a graph with 
no simple circuits containing fewer trees than were present 
before this step; see Exercise 24.) Continue the process of 
simultaneously adding edges connecting trees until n - 1 
edges have been chosen. At this stage a minimum spanning 
tree has been constructed. 

*24. Show that the addition of edges at each stage of Soilin'* 
algorithm produces a forest. 

25, Use Sollm's algorithm to produce a minimum spanning 
tree for the weighted graph shown in 
a) Figure I. b) Figure 3. 

*26. Express Sulim's algorithm in pseudocode. 

**27. Prove that Soilin'* algorithm produces a minimum 
spanning tree in a connected undirected weighted 
graph. 

*28. Show that the first step of Solim's algorithm produces 
a forest containing at least f/i/S] edges. 

*29, Show that if there are r trees in the forest at some in¬ 
termediate step of Soilin'* algorithm, then at least |V/2] 
edges arc added by the next iteration of the algorithm. 

*30. Show that no more than [<j/2*J trees remain after the 
first step of Soliin's algorithm has been carried out and 
the second step of the algorithm has been carried out 
k - 1 times. 

*31. Show that Soilin'* algorithm requires at most log n it¬ 
erations to produce a minimum spanning tree from a 
connected undirected weighted graph with n vertices. 

32. Prove that Kruskal's algorithm produces minimum 
spanning trees. 
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Key Terms and Results 

TERMS 

tree: u connected undirected graph with no simple circuits 
forest: an undirected graph with no simple circuits 
rooted tree: a directed graph with a specified vertex, 
tailed the root, such that there is a unique path to any 
other vertex from this root 
subtree: a subgraph of a tree that is also a tree 
parent of v in a rooted tree: the vertex u such that ( «, v) 
is an edge of the rooted tree 

child of a vertex v in a rooted tree: any vertex with v as 
its parent 

sibling of a vertex v in a rooted tree: a vertex with the 
same parent as v 

ancestor of a vertex v in a rooted tree: any vertex on the 
path from the root to c 

descendant of a vertex v in a rooted tree: any vertex that 
has r as an ancestor 

internal vertex: a vertex that has children 
leaf: a vertex with no children 

level of a vertex: the length of the path from the root to 
this vertex 

height of a tree: the largest level of the vertices of a tree 
m-ary tree: a tree with the property that every internal ver¬ 
tex has no more than m children 
full m-ary tree: a tree with the property that every internal 
vertex has exactly m children 

biliary tree: an /N-ary tree with m = 2 teach child may be 
designated as a left or a tight child of its parent) 
ordered tree: a tree in which the children of each internal 
vertex tire linearly ordered 

balanced tree: a tree in which every vertex is at level /tor 
h - 1. where h is Lhe height of the tree 
binary search tree: a binary tree in which the vertices arc 
labeled with items so that a label of a vertex is greater 
than the labels of all vertices in the left subtree of this 
vertex and is less than the labels of all vertices in the 
right subtree of this vertex 

decision tree: a rooted tree where each vertex represents a 
possible outcome of a decision and the leaves represent 
the possible solutions 

prefix code: a code [bat has the property that [he code of a 
character is never a prefix of the code of another character 
tree traversal: a listing of the vertices of a tree 
preorder traversal: a listing of the vertices of an ordered 
rooted tree defined recursively by specifying that the root 
is listed, followed by the first subtree, followed bv the 
other subtrees in the order they occur from left to right 
inorder traversal; a listing of the vertices of an ordered 
Moled tree defined recursively by specifying that the first 
subtree is listed, followed by Lhe root, followed by the 
other subtrees in the order they occur from left to right 
poslorder traversal: a listing of the vertices of an ordered 


routed tree defined recursively by specifying that the sub¬ 
trees arc listed in the order they occur from left to right, 
followed by the root 

infix notation: the form of art expression (including a full 
set of parentheses) obtained from an inorder traversal of 
the binary tree representing this expression 
prefix (or Polish) notation: the form of an expression ob¬ 
tained from a preorder traversal of the tree represen!mg 
this expression 

postfix (or reverse Polish) notation: the form of an ex¬ 
pression obtained from a postorder traversal of the dec 
representing this expression 

sorting problem: a problem in which a list of items is to 
be pul into increasing order 

spanning tree: a tree containing all vertices of a graph 
minimum spanning tree: a spanning tree with smallest 
possible sum of weights of its edges 
greedy algorithm: an algorithm that optimizes by making 
the optimal choice at each step 

RESULTS 

A graph is a tree if and only if there is a unique simple path 
between any nf its vertices, 

A tree with n vertices has n - t edges. 

A full m-arv tree with / internal vertices has mi - 1 vertices. 
The relationships between the numbers of vertices, leaves, 
and internal vertices in a full rn-ary tree (see Theorem 4 
in Section 8.1 ), 

There tux; a: most m“ leaves in an m-ary tree of height fi¬ 
lial) m-ary tree hast leaves, its height ft is at least flog m /]. if 
(he tree is also foil and balanced, then its height isifog ra l i. 
the bubble sort: a sorting procedure that is carried out us¬ 
ing passes where successive items that are out of order 
arc interchanged 

the merge sort: a sorting procedure that is carried out by 
successively merging pairs of sublists of the original list 
depth-first search, or backtracking: aprocedure for con¬ 
struct mg a spanning tree by adding edges that form a path 
until this is nut possible, and then moving back up the path 
until a vertex is found where a new path tan be formed 
breadth-first search: a procedure for constructing a spun 
rting tree that successively adds all edges incident to the 
last set of edges added, unless a simple circuit is formed 
Prim’s algorithm: a procedure for producing a minimum 
spanning tree in a weighted graph that successively adds 
edges with minimal weight among all edges incident to 
u vertex already in the tree such that no edge produces a 
simple circuit when it is added 
Jvruskal’S algorithm: a procedure for producing a mini ¬ 
mum spanning tree in a weighted graph that successively 
adds edges of least weight that are not already in the tree 
such that □ o edge prod uc e s a si mpfe ci rcsiit when it is added 
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Review Questions 


1, a) Define a tree, b) Define a forest. 

2, Can there be two different simple paths between the 
vertices of a tree 1 ’ 

3, Give at least three examples of how trees are used in 
modeling 

4, a) Define a rooted tree and the root of such a tree. 

b) Define the parent of a vertex and a child of a vertex 
in a rooted tree, 

e> What are an imenial vertex, a leaf, and a subtree in 
a rooted tree‘l 

d) Draw a rooted tree with at least 10 vertices, where 
the degree of each vertex does not exceed 3. Iden¬ 
tify the root, the parent of each vertex, the children 
of each vertex, the interna! vertices, and the leaves. 

5, a ) I low many edges does a tree with n vertices have'/ 
b) What do you need to know to determine the number 

of edges in a forest with n vertices? 

0. a) Define a full m-ary tree, 

b) How many vertices does a full m-ary tree have if it 
has i internal vertices'? How many leaves does the 
tree have ? 

7. at What is the height of a rooted tree? 
b) What is a balanced tree? 

et How many leaves can an m-ary tree of height h 

have? 

8. a) What is a binary search tree? 1 

b) Describe an algorithm for constructing a binary 
search tree. 

c) Form a binary search tree for the words vireo, wetr- 
btcK egret, grosbeak, nuthatch, and kingfisher. 

9. a) What is a prefix code'.’ 

b) How can a prefix code be represented by a binary' 
tree? 

10. ai Define preorder, inorder, and postorder tree traver¬ 

sal. 

b) Give an example of preorder, postorder, and inorder 
traversal of a binary tree of your choice with at least 
12 vertices, 

11, a) Explain how to use preorder, inorder, and postorder 

traversals to find the prefix, infix, and postfix forms 
of an tin th me tie expression, 
b) Draw the ordered rooted tree that represents ((.r - 
3)•' *.i • u ri'ui. 


c) Find the prefix and postfix forms of the expression 
in part (b). 

12. Show that the number of comparisons used by a sorting 
algorithm is at least dog n!]. 

13. a) Describe the bubble sort algorithm. 

bl Use the bubble sort algorithm to put the list 5, 2, 4. 
1,3 in increasing order. 

c) Give a big-0 estimate for the number of compar¬ 
isons used by the bubble sort 

14. ai Describe the merge sort algorithm. 

b) Use the merge sort algorithm to put fits list. 4.10, 1, 
5, 3, ft. 7, 2, ft. 9 in increasing order. 

c) Give a big-0 estimate for the number of compar¬ 
isons used by the merge sort. 

15. ai What is a spanning tree of a simple graph? 

b) Which simple graphs have spanning trees? 1 

c) Describe at least two different applications that 
require that a spanning tree of a simple graph be 
found. 

16. a) Describe two different algorithms for finding a 

spanning tree in a simple graph, 
b) Illustrate how the two algorithms you described in 
(a) can be used to find the spanning tree of a simple 
graph, using a graph of your choice with at least 
eight vertices and 15 edges. 

17. a) Explain how backtracking can be used to determine 

whether a simple graph can be colored using n col¬ 
ors. 

b) Show, with an example, how backtracking can be 
used to show' that a graph with a chromatic number 
equal to 4 cannot be colored with three colors, but 
can be colored with four colors. 

18. a) What is a minimum spanning tree of a connected 

weighted graph? 

b) Describe at leas! two different applications that 
require that a minimum spanning tree of a con¬ 
nected weighted graph be found. 

19. a) Describe Kruskal’.x algorithm and Prim’s algorithm 

for finding minimum spanning trees, 
h) Illustrate how Kruskal’s algorithm and Prim's 
algorithm are used to find a minimum spanning 
tree, using a weighted graph with at least eight 
vertices and 15 edges. 


Supplementary Exercises 


* 1 . Show that a simple graph is a tree if and only if it con¬ 
tains no simple circuits and the addition of an edge 
connecting two nonadjacent vertices produces a new 


graph that has exactly one simple circuit (where cir¬ 
cuits that contain the same edges are not considered 
different). 
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*2. How many non isomorphic rooted trees are there with 
si* vertices? 

3. Show that every tree with at least one edge must have 
at least two pendant vertices. 

4. Show that a tree with n vertices which has n - I 
pendant vertices must he isomorphic to A*i ( . 

5. What is the sum of the degrees of the vertices of a tree 
with 7i vertices? 

*6. Suppose that d], d lt .,,, d n are a positive integers with 
sum 2n - 2. Show that there is a tree which has 
n vertices so that the degrees of these vertices are 
d t .d z . <1„. 

7. Show that every' tree is a planar graph 

8. Show that every tree is bipartite. 

9. Show' that every' forest can be colored using two colors. 

^ A B-free of degree k is a rooted tree such that all its leaves 
are at the same level, its root has at least two and at most k 
children unless it is a leaf, and every interna! vertex other 
than the root has at least ft/2|, but no more than k, children. 
Computer files can be accessed efficiently when B-trees are 
used to represent them. 

10. Draw three differe nt B-trees of degree 3 with height 4. 
*11. Give an upper bound and a lower bound for the number 
of leaves in a B-tree of degree k with height h. 

*12. Give an upper bound and a low er bound for the height 
of a B-tree of degree k w ith ft leaves. 

A rooted tree I is called an Si-tree if it satisfies the follow¬ 
ing recursive definition. It is an So-tree if it has one vertex. 
For k > 0, T is an 5*-tree if it can be built from two S k t - 
trees by making the root of one the root of the 3Vtree and 
making the root of the other die child of the root of the first 
Si -tree. 

13. Draw an .Si-tree for k ~ 0, L 2. 3.4, 

14. Show that an 5* -tree has 2* vertices and a unique 
vertex at level k. This vertex at level k is called She 

handle. 

*15. Suppose that T is an Sf-tree with handle v. Show that 

T can be obtained from disjoint trees r<>, T ] .7V- i, 

where v is not in any of these trees, where T, is an .V,- 

tree tor / = 0,1_ _ k - 1, by connecting v to r ( , and 

r, to r i+ [ for i = 0, 1_ k - 2. 

"Die listing of the vertices of an ordered rooted tree in level 
order begins with the root, followed by the vertices at level 
1 from left to right, followed by the vertices at level 2 from 
left to right, and so on. 

16. List the vertices of the ordered rooted trees in Figures 
3 and 9 of Section 8.3 in level order, 

17. Devise an algorithm for listing the vertices of an or¬ 
dered rooted tree in level order, 

*J 8, Devise an algorithm for determining if a set of univer¬ 
sal addresses can he the addresses of the leaves of a 
rooted tree. 


19. Devise an algorithm for construe ting a rooted tree iron: 
the universal addresses of its leaves. 

The insertion, sort operates by considering the eletncrs 
a list one at a time, beginning with the second elemero . r. 
element is compared to the previous elements in the ii>;. 
which have been put in the correct order, and this element i-, 
put in the correct position among these, moving the dome m 
that was in this position, and all elements to the rind !*( ;hk. 
one position to the right. 

20. Sort the list 3,2,4,5, I using an insertion so"; 

21. Write the insertion sort in pseudocode. 

22. Determine the worst-case complexity of the —i-.-a 
sort in terms of the number of comparisons uv.i. 

23. Suppose that e is an edge in a simple graph Lhr !• 
cident to a pendant vertex. Show that e must be -r aro 
spanning tree. 

A cut set of a graph is a set of edges such that the remov a; 
of these edges produces a subgraph with more connected 
components than in the original graph, but no proper stibstN 
of this set of edges has (has property. 

24. Show that a cut set of a graph must have at feast one 
edge in common w ith any spanning tree of rhi:t graph 

A cactus is a connected graph in which no edge is in m<ro. 
than one simple circuit not passing through an v vertex orti,;.' 
than its initial vertex more than once or its initial vcric • 
other than at its terminal vertex (where two drams 
contain the same edges are tint considered different t 

25. Which of the following graphs are cacti? 
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26. Is a tree necessarily a cactus? 

27. Slum that a cactus is formed if we add a circuit con- 
laming new edges beginning and ending at a vertex of 
a tree. 

*28. Slum' that if every circuit not passing through any ver¬ 
tex other than its initial vertex more than once in a 
connected graph contains an odd number of edges, then 
this graph must tv a cactus. 



A i 


A degree-constrained spanning tree of a simple graph 
G is a spanning tree with the property that the degree of 
a vertex in [his tree cannot exceed some specified bound. 
Degree-ennstrained spanning trees are useful in models of 
transportation systems where the number of roads at an in¬ 
tersection is limited, models of communications networks 
where the number of links entering a node is limited, and 
so on. 

In Exercises 2d-.s J iiod a degree-constrained spanning tree 
Of tile given graph where each vertex has degree less than 
or equal in 3. or show that such a spanning tree does not 
exist. 


2D. 




\ 


d ■ --*3 — 


d) 


A caterpillar is a tree that contains a simple path such that 
every vertex not contained in t his path is adjacent to a vertex 
in the path. 

34. Which of the graphs in Exercise 33 are caterpillars? 

35. How many non isomorphic caterpillars are there with 
six vertices',' 

*36. a) Prove or disprove that all trees whose edges form a 
single path are graceful. 

*M>) Prove or disprove that all caterpillars are graceful. 
37. Suppose that the first four moves of a tie-tae-toe game 
arc as shown. Explain how a tree can be used to show 
the possible successive moves of this game. If the 
player using X goes first, does this player have a strat¬ 
egy that will always win? 




V K ~i 



ft ,* 


32. Show that a degree-constrained spanning tree of a sim¬ 
ple graph in which each vertex has degree not ex¬ 
ceeding 2 consists of a single Hamilton path in the 
graph. 

33. A tree with n vertices is called graceful if its vertices 
can be labeled with the integers 1,2,.... n such that the 
absolute value of the difference of the labels of adja¬ 
cent vertices are all different. Show that the following 
trees me graceful, 


38. Three couples arrive at the bank of a river. Each of the 
wives is jealous and does not trust her husband when he 
is with one of the other wives (and perhaps with other 
people), but not with her. How can six people cross to 
the other side of the river using a boat that can hold 
no more than two people so that no husband is alone 
with a woman other than his ,wife? Use a graph theory 
model. 

*39. Suppose that e is an edge in a weighted graph that is 
incident to a vertex v so that the weight of e does not 
exceed the weight of any other edge incident to v. Show 
that there exists a minimum spanning tree containing 
this edge. 

*40. Show- that if no two edges in a w eighted graph have the 
same weight, then the edge with least weight incident 
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to a vertex v is included in every minimum spanning 
tree. 

41. Kind a minimum spanning tree of each of the following 
graphs where the degree of each vertex in the spanning 
tree does not exceed 2. 

a) , b) 




Computer Projects 

WRITE PROGRAMS WITH THE EOT LOWING INPIT AND OUTPUT. 


1. Given the adjacency matrix of an undirected simple 
graph, determine whether the graph is a tree. 

2. Given the adjacency maim of a rooted tree and a ver¬ 
tex in the tree, find the parent, children, ancestors, de¬ 
scendants, and level of this vertex. 

3. Given the list of edges of a rooted tree and a vertex 
in the tree, find the parent, children, ancestors, descen¬ 
dants, and level of this vertex. 

4. Given a list of items, construct a binary search tree con¬ 
taining these items. 

5. Given a binary search tree and an item, locate or add 
this item to the binary 1 2 search tree. 

6. Given the ordered list of edges of an ordered rooted 
tree, find the universal addresses of its vertices 

7. Given the ordered list of edges of an ordered rooted 
tree, list its vertices in preorder, in order, and post¬ 
order, 

8. Given an arithmetic expression in prefix form, find its 
value, 

9. Given an arithmetic expression in postfix form, find its 
value. 

10. G iven a set of a integers, sort them using a bubble sort. 

11. Gi ven two sorted lists of integers, merge them into one 


sorted list, keeping track of the number of comparisons 
used. 

12. Given a set of n integers, sort them using a merge sort. 

13. Given the adjacency matrix of a connected undirected 
simple graph, find a spanning tree for this graph using 
a depth-first search. 

14. Given the adjacency matrix of a connected undirected 
simple graph, find a spanning tree for this graph using 
a breadth-first search. 

15. Given a set of positive integers and a positive integer 
jV. use backtracking to find a subset of these integers 
that have ,V as their sum. 

*16. Given the adjacency matrix of an undirected simple 
graph, use backtracking to color the graph with three 
colors, if this is possible. 

*17. Given a positive integer n, solve the u-queens problem 
using backtracking, 

IS, Given the list of edges and their weigh Is of a weighted 
undirected connected graph, use Prim s algorithm to 
find a minimum spanning tree of this graph. 

19, Given the list of edges and their weights of a weighted 
undirected connected graph, use Kruskal’s algorithm to 
find a minimum spanning tree of this graph. 


Computations and Explorations 


I 'F A t'tJMPl I TATI ON AT PROGRAM OR PROGRAMS YOU HAV 


1. Display all trees with six vertices. 

2. Display a full set of nonisomorphic trees with seven 
vertices. 

*3. Construct a Huffman code for the letters of the English 


F, WRITTEN TO DO THE FOLLOWING EXERCISES. 

language based on the frequency of their occurrence in 
ordinary English text. 

4, Compute the number of different spanning trees of K„ 
for n - 1.2,3,4,5,6, Conjecture a formula for the 
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number o I' such spanning, trees whenever n is a positive 
inleqer 

Comjmre the it umber of comparisons needed to 
sort lists of n elements tin n = 100, iOQG, and 
0,000 where the elements are randomly selected 
positive integers, using the selection sort, the 
insertion sort, the merge sort, and the quick 

ii'rs. 

Cittuputt: the number of different ways n queens cart be 


arranged on an n x « chessboard so that no two queens 
can attack each other for all positive integers n not ex¬ 
ceeding 10. 

*7. Find a minimum spanning tree of the graph that con¬ 
nects the capital cities of the fifty stator ;n the U.S. to 
each other whore the weight of each edge is the distance 
between the cities. 

8, Draw the complete game tree for a game of checkers 
on a 4 X 4 board. 


Writing Projects 

Ki.xmNDTO i; ! FOLLOW INC WITH I SS A' • 1 S!N'G OCT-SIDE SOURCES. 


1. Explain how Cayley used trees to enumerate the num¬ 
ber of certain types of hydrocarbons. 

2. Define AVIstmts {sometimes also known as heigkt- 
tet!anted trees). Describe how and why AVL-trees are 
used in a variety of different algorithms. 

3. Define quad 1 trees and explain how images CM be rep¬ 
resented using them. Describe how images can be ro¬ 
tated scaled, and translated by manipulating the corre¬ 
sponding quad tree. 

■ 5. Dcline a ftcetp and explain how trees can be turned into 
heaps. Why are heaps useful in sorting? 

5, Describe algorithms for data compression based on let¬ 
ter frequencies, including Huffman coding, and related 
algorithms based on frequencies of blocks of letters, 

6. Discuss how trees are used to model games and to de- 
vclop winning strategies for games. Explain how to 
-'tide a game, such as tic-tac-tpe, nim, hex, or some 


other game, using game trees. Be sure to discuss alpha- 
beta pruning. 

7. Define the type of graph known as a mesh of trees. 
Explain how this graph is used in applications to very 
large system integration and parallel computing. 

8. Discuss the algorithms used in IP multicasting to avoid 
loops between routers, 

9. Describe an algorithm for finding the minimum span¬ 
ning tree of a graph so that the maximum degree of any 
vertex in the spanning tree does not exceed a fixed con¬ 
stant k. 

10. Compare and eonstrast some of the most important sort¬ 
ing algorithms in terms of their complexity and when 
they are used. 

11. Discuss the history and origins of algorithms for con¬ 
structing minimum spanning trees. 

12. Describe algorithms for producing random trees, 
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T he circuits in computers and other electronic devices have inputs, each of which 
is either a 0 or a 1, and produce outputs that are also Os and Is. Circuits can 
be constructed using any basic element that has two different states, Such ele¬ 
ments include switches that can be in either the on or the off position and optical devices 
that can either be lit or unlit. In 1938 Claude Shannon showed how the basic rules of 
logic, lirst given by George Boole in 1854 in his The Imws of Though!, could be used 
to design circuits. These rules form the basis for Boolean algebra. In this chapter we 
develop the basic properties of Boolean algebra. The operation of a circuit is defined 
by a Boolean function that specifies the value of an output for each set of inputs, The 
first step in constructing a circuit is to represent its Boolean function by an expression 
built up using the basic operations of Boolean algebra. We will provide an algorithm 
for producing such expressions, The expression that we obtain may contain many more 
operations than are necessary to represent the function. Later in the chapter we will 
describe methods for finding an expression with the m ini mum number of sums and 
products that represents a Boolean function, The procedures that we will develop, Kar¬ 
naugh maps and (he Quinc-VkCluskey method, are important tn the design of efficient 
circuits. 


9 A _ 

Boolean Functions 

INTRODUCTION 


Boolean algebra provides the operations and the rules for working with the set {(), If 
■Electronic and optical switches can be studied using this set and the rules of Boolean 
algebra. The three operations in Boolean algebra that we will use most are comple¬ 
mentation. the Boolean sum, and the Boolean product. The complement of an element, 
denoted with a bar. is defined by 0 - 1 and I - (J. The Boolean sum. denoted by 1 or 
by OR. has the following values: 

1-1-1, I 0 - I, h i I 0 + 0 - 0, 

The Boolean product, denoted by • or by AND. has the following values: 

1 ■ I - l, 1-0-0, 0-1=0, 0-0 = 0. 

When there is no danger of confusion, the symbol ■ can be deleted, just as in writing 
algebraic products. Unless parentheses are used, the rules of precedence for Boolean 
operators are: first, all complements are computed, followed by Boolean products, fol¬ 
lowed by all Boolean sums. This is illustrated in Example 1. 

593 
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EXAMPLE J Find the value of I ■ 0 + t() + I}. 

Solution: Using the definitions, of complementation, the Boolean sun), ami the Boolean 
product, it follows that 

(1 ■(») +(Q • 11 - 0+1 

- tj -t 0 

= a ■ 


The complement. Boolean sum, and Boolean product correspond to the logical op¬ 
erators, V'- and A, respectively, where 0 corresponds to F {false.) and I corresponds 
to T (true). The results of Boolean algebra can be directly translated imo resuhs about 
propositions. Conversely, results about propositions can be translated into statements 
about Boolean algebra. 


BOOLEAN EXPRESSIONS AND BOOLEAN FUNCTIONS 


| TABLE 1 

X 

y | 

Fix. yi 

1 | 

i 

0 

1 

0 

1 

0 i 

i , 

0 

111 

o 1 

0 

J 


Let P = {0, 1}. The variable x is called a Boolean variable if it assumes values only 

from B. A function from B n , the set jf.i j, _,r„) | jt,- £ B, 1 i s n}, to B is called 

a Boolean function of degree n. The values of a Boolean function are often displayed 
in tables. For instance, the Boolean function Fix. y) with the value 1 when x = 1 and 
y = 0 and the value 0 for ail other choices of .r and y can be represented by Table 1, 
Boolean functions can be represented using expressions made up from the variables 

and Boolean operations. The Boolean expressions in the variables x\, xj,_,r„ are 

defined recursively as follows'. 

0, I, Ay, + 2 , — x„ are Boolean expressions; 

if E\ and £2 are Boolean expressions, then £ ; , (£ 1 £ 1 2 ), and (£| + £ 2 ) are Boolean 
expressions. 

Each Boolean expression represents a Boolean function. The values of this function are 
obtained by substituting 0 and I for the variables in the expression. In Section 9.2 we 
will show that every Boolean function can be- represented by a Boolean expression. 


(. iaiide t, I wood Shannon (.born 1916), Claude Shannon was bom in Gaylord, Michigan, and attended 
the University <4 Michigan, graduating in IW(i, He continued his studies at M.t.T., where he took the job 
Wfb ° f ' “ in ^ differential analyser, a mechanical computing device consisting of shafts and gears built 
by his professor, Vtinncvar Bush. Shannon's master's thesis, written in 1936, studied the logical aspects 
of the differential analyzer. This master’s thesis presents the first application of Boolean algebra to the 
design of switching circuits; it is perhaps the most famous master's thesis of (tie twentieth century. He 
received his fibi.D. from M.t.T. m 1940. Shannon joined Bell Laboratories in 1940. where he worked on 
transmitting data efficiently. He was one yf the first people to use bits to represent information. At Bell 
Laboratories he worked on determining Lbe amount of traffic that telephone lines can carry. Shannon made 
many fundamental ranrnbutions in information theory. In the early 1950s he’ was one of the founders of 
the study of artiliciai intelligence. He joined I be M.t.T. faculty in 1956, where he continued his study of 
iiifiwnfliiuFi theory 

.Shannon !us jti:,. ip. i entitma! sniv. i If is credited w : ith inventing the rocket-powered frisbec. He is 
also luffitiui for itiiifig u umcyde down the hallway < of Beil Laboratories white juggling four balls. Shannon 
retired when he was 50 years hid, publishing papers sporadically over the following 10 years. Currently, he 
concentrates on enjoying life and working on some pet projects, such as building a motorized pogn stick. 
One unenesring quote tkup puWLlied in Omni Magazine in 1987. is "I visualize a time when we 

vcii; be m r ■•'■.-tv wu;*! hwrcrikr At if ! am moling for the machines.” 
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K.VAMP1.K 2 f ind [he values of the Boolean function represented by F{x, y, ;) = xy * 

SulutuMi: The values of this function are displayed m Table 2. ■ 


The Boolean funclions F and 6' of n variables are equal if and only if t\h x , 

h i—, b n ) - G(h],h 2 . b„) whenever b],bi, ...,b n belong to B, Two different 

Boolean expressions that represent the same function are called equivalent. For in¬ 
stance, all the Boolean expressions xy, xy + 0, and xy ■ 1 are equivalent. The comple¬ 
ment of the Boolean function F' is the functionwhere F(xi . x„) - Fix i,, 

Let F and G be Boolean functions of degree n. The Boolean sum F 4 G and Boolean 
product FG are defined by 

(F + G){X[. x„) = Fix i . x„) + Cj(x i _x„), 

:F(7); t.tj = F{x i,.,., x n )G{X\,.x n ). 

A Boolean function of degree 2 is a function from a set with four dements, namely, 
pairs of elements from B = {0, 1}, to B, a set with two elements. Hence, there are 16 
different Boolean functions of degree 2. In Table 3 we display the values of the 16 
different Boolean functions of degree 2, labeled F \. F\,... 
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EXAMPLE 3 


EXAMPLE 4 


TABLE 4 The Number of Boolean 
Functions of Degree n. 

Degree 

Number 

1 

4 

7 

16 

3 

256 

4 

65,536 

5 

4,294.967,296 

6 

18,446,744,073,709,551,616 


How many different Boolean functions of degree n are there'/ 


Solution: From the product rule for counting, it follows that there are 2" different 
/i-tuples of Os and Is. Since a Boolean function is an assignment of 0 or I to each of 
these 2" different n-tupies, the product rule shows that there are 2^” different Boolean 
functions, ■ 


Table 4 displays the number of different Boolean functions of degrees I through 6. 
The number of such functions grows extremely rapidly. 


IDENTITIES OF BOOLEAN ALGEBRA 

There arc many identities in Boolean algebra. The most important of these are displayed 
in Table 5. (The reader should compare these identities to the logical equivalences in 
Table 5 of Section 1.2 and the set identities in Table 1 in Section 1.5. All are special 
cases of the same set of identities in a more abstract structure.) These identities are 
particularly useful in simplifying the design of circuits. Each of the identities in Table 
5 can be proved using a table. We will prove one of the distributive law's in this way in 
the following example. The proofs of the remaining properties are left as exercises for 
the reader. 


Show that the distributive law jc{ y + ;) = xy 4 xz is valid. 

Soiuhon: The verification of this identity is shown in Table 6, The identity holds be¬ 
cause the last two columns of the table agree. 0 


The identities in Table 5 can be used to prove further identities. We demonstrate this in 
the following example. 
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TABLE 5 Boolean Identities, 

Identity 

Name 

X -- X 

Law of the double complement 

x + x = .r 

A ' X - X 

Idempotent laws 

x + 0 « x 

.V 1 = V 

Identity laws 

.( + 1 -^ ! 

V -0-0 

Dominance laws 

JT + V - V + X 

xy = yx 

Commutative laws 

-v 4 (V + z) = (X 4 J) 4 - 
x(yz) = (xy)x 

Associative law's 

.r + yz = (x + y)(x + z) 

,t(y 4 ;) ~ XV 4 xz 

Distributive laws 

( .tv) = x 4 T 

De Morgan's law's 

(x 4 y ) xy 



TABLE 6 

Jt 

■V 

\ 

z 

y + z 

xy 

xz 

x(y 4 z) 

xy + xz 

1 

1 

1 

1 

l 

0 

1 

1 

1 

1 

I 

0 

1 

1 

1 

1 

1 

0 

1 

1 

0 

1 

l 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

0 

0 

0 

0 

(1 

1 

0 

1 

0 

0 

0 

0 

o 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 
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EXAMPLE 5 


EXAMPLE ft 


EXAMPLE 7 


Prove the absorption law a(x + y) = x using the identities of Boolean algebra. (This 
is called an absorption law since absorbing x t- y into x leaves x unchanged.) 


Solution: The steps used to derive this identity and the law used in each step follow: 


r(.t + y) = (1 -f ())(x + y) 
= x + 0 • y 

-.try-0 
- JC + 0 


Identity law for the Boolean sum 
Distributive law of the Boolean sum over the 
Boolean product 

Commutative law for the Boolean product 

Dominance law lor the Boolean product 

Identity law for the Boolean sum ■ 


DUALITY 


The identities in Table 5 come in pairs (except for the law of the double complement). 
To explain the relationship between the two identities tn each pair we use the concept 
of a dual. The dual of a Boolean expression is obtained by interchanging Boolean sums 
and Boolean products and interchanging Os and I s. 


Find the duals Of ,v(y +- 0) and .x - I 4- fy + 

Solution: Interchanging ■ signs and + signs and interchanging Os and Is in these ex¬ 
pressions produces their duals. The duals are .t 4 (y ■ n and (x + O K vzK respectively, 


The dual of a Boolean function !■ represented by a Boolean expression is the func¬ 
tion represented by the dual of this expression. This dual function, denoted bv F fi , does 
not depend on the particular Boolean expression used to represent F. An identity be¬ 
tween functions represented by Boolean expressions remains valid when the duals of 
both sides of the identity are taken. (See Exercise 22 for the reason this is true.) This 
result, called the duality principle, is useful for obtaining new identities. 


Construct an identity from the absorption law ,v( v + v) = j given in Example 5 by 
taking duals. 

Solution: Taking the duals of both sides of this identity produces the identity x +- X v = 
x, which is also called an absorption law. I 


THE ABSTRACT DEFINITION OF A BOOLEAN ALGEBRA 


In this section we have focused on Boolean functions and expressions. However, the 
results we have established can be translated into results about propositions or results 
about sets. Because of this, it is useful to define Boolean algebras abstractly. Once it is 
shown that a particular structure is a Boolean algebra, then all results established about 
Boolean algebras in general apply to this particular structure, 

Boolean algebras can be defined in several ways. The most common way is to 
specify the properties that operations must satisfy, as is done in the following definition. 
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DEriNlllON 1, AEoflteinalgebra isasetg withtiiwWnary opetaticft^yand 
A, etettigftf 0 ss«J 1, and a unaiy operation ~ such thafthe faltowingproperties 
bold for all x* y, aadz i n B: 


x\jO = x\ 
x A J = X 


Identity laws 


jcVx*= l] 
jtAj-Dj 


Domination laws 


(*Vy)Vz = x\/{y\Jz)\ 
(xAy)Az = xA(y Az)j 


Associati ve laws 


jrVy - y V*j 

x Ay = yAjtj 


Commutative laws 


*V(yAz) = (x \j y) A{x\/ -)[ 
x A (y V z) = (x A y) V (x A z)j 


Distributive laws 


Using the laws given in Definition 1, it is possible to prove many other laws that hold 
for every Boolean algebra, such as idempotent and dominance laws. (See Exercises 
25-32.) 

From our previous discussion, 8 — {0,1} with the OR and AND operations and the 
complement operator, satisfies all these properties- The set of propositions in n vari¬ 
ables, with the V and A operators, F and T, and the negation operator, also satisfies all 
the properties of a Boolean algebra, as can be seen from Table 5 in Section 122. Simi¬ 
larly, the set of subsets of a universal set V with the union and intersection operations, 
the empty set and the universal set, and the set complementation operator, is a Boolean 
algebra as can be seen by consulting Table I in Section 1,5. So, to establish results about 
each of Boolean expressions, propositions, and sets, we need only prove results about 
abstract Boolean algebras. 

Boolean algebras may also be defined using-the notion of a lattice, discussed in 
Chapter 6. Recall that a lattice L is a partially ordered set in which every pair of elements 
x, y has a least upper bound, denoted by lub(x, y) and a greatest lower bound denoted 
by glbu, y). Given two elements x and y of L, we can define two operations V and A 
on pairs of elements of L by x y v = lub(x, y) and x A y = glb(x, y). 

For a lattice L to be a Boolean algebra as specified in Definition 1, it must have two 
properties, First, it must be complemented. For a lattice to be complemented it must 
have a least dement 0 and a greatest element 1 and for every element x of the lattice 
there must exist an dement ~x such that x y x - 1 and x A i - 0. Second, it must be 
distributive. This means that for every x, y, and zinl, x y (y Az) = (jry y) A (x y z) 
and .r A (y y z) - (x Ay) y (x A z). Showing that a complemented, distributive lattice 
is a Boolean algebra is left as Exercise 33 at the end of this section. 


Exercises 


1. Find the values of the following expressions. 

a) 10 b) I + 1 c) 0 ■ 0 d) (l + flj 

2, Find the values, if any, of the Boolean variable v that 


satisfy the following equations 
a) x ■ t = 0 b) x + x = 0 

c) x • 1 » x d) x ■ x = 1 
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3. What values of the Boolean variables x and y satisfy 
xy — x + v? 

4. How many different Boolean functions are there of 
degree 7 ? 

5. Prove the absorption law ,r+ xy = x using the laws 
in Table 5. 

6. Show that F(x, y, r ) = .tv + jcz + y- has the value 1 
if and only if at least two of the variables x, y, and z 
have the value 1. 

7. Show that ,ry + y; -t- xz = xy + y: + xz. 

Exercises 8-15 deal with the Boolean algebra defined by 
the Boolean sum and Boolean product on {0, I}, 

8. Verify the law of the double complement. 

9. Verify the idempotent law s. 

10. Verify the identity laws. 

11. Verify the dominance laws. 

12. Verify the commutative laws. 

13. Verify the associative laws. 

14. Verify the first distributive law in Table 5. 

15. Verify De Morgan’s laws. 

The Boolean operator®, called the XOR operator, is defined 
by 1 ® I = 0. I ®0 - 1. 0® I = l,and0®0 - (}. 

16. Simplify the following expressions. 

a) x 6 0 b) x © 1 c) ,t ffi x d) x © 3 

17. Show that the following identities hold. 

a) .r © y - ;(,r + yj( ivl 

b) x © y = (,ry)+ (xy) 

18. Show that .r © y = v © ,r. 

19. Prove or disprove the following equalities. 

a) a©(>■©?> = (jr®y>®j 

b) X 4 ( V Cp = U + y) © l.v + 

C) JC©(>' + ;) ■= ufr vj r :.V0C) 

20. Find the duals of the follow ing Boolean expressions, 

a) x + y b) xy 

c) x y ; + jc yz d) xl + x ■ f) + x - 1 

*21. Suppose that F is a Boolean function represented 

by a Boolean expression in the variables jti,_r fl . 

Show that F d lx t , .t„) - Ffx^T.T. £„). 

*22. Show that if F and G are Boolean functions repre- 

9.2 


settled by Boolean expressions in n variables and 
F = G, then F d = G d t where F' ! and G'' : are the 
Boolean functions represented by the duals of [he 
Boolean expressions representing F ;md G, respec¬ 
tively, {Hint: Use the result of Exercise 21.) 

*23. How many ditferem Boolean functions Fi x, y, -) are 
there so that Fix, y, z) = F (.r, y, z) for ail values of 
tlie Boolean variables x, y. and z? 

*24. How many different Boolean functions Fix, y, i) are 
there so that F(x, y, -) = Fix, y, z) - fix, y, j) for 
all values of the Boolean variables x, v,;? 

In Exercises 25 -32, use the laws in Definition I to show- 
that the stated properties hold in every Boolean algebra. 

25. Show that in a Boolean algebra, the idempotent 
properties x y x ~ x and x A x - x hold for 
every element x- 

26. Show that in a Boolean algebra, every' element x has 
a unique complement jr such that « y x - 1 and 
x A , K = 0 , 

27. Show 1 that in a Boolean algebra, the complement of 
the element Q is the element I and vice versa. 

28. Prove that in a Boolean algebra, the law of the dou* 
ble complement holds; that is, x = x for every' ele¬ 
ment x. 

29. Show- 1 that De Morgan’s laws hold in a Boolean alge¬ 
bra. That is, shew that for alia andy. (r y v) “ xAy 
and (j A y) = ,v y v. 

30. Show that in a Boolean algebra, Lhe modular prop¬ 
erties hold. That is, show that x A (y V (-f A 4) = 
(x A y) \j (x A z) and x y fy A (r y j)) = 
UVy)A (xy 4, 

31. Show that in a Boolean algebra, if r y v - 0, then 
x = 0 and v = f) and that if x A y = 1, then x = 1 
andy = 1. 

32. Show that in a Boolean algebra, the dual of an iden¬ 
tity', obtained by interchanging the y and A opera¬ 
tors and interchanging the elements 0 and I, is also 
a valid identity, 

33. Show' that a complemented, distributive lattice is a 
Boolean algebra. 


Representing Boolean Functions 

two important problems of Boolean algebra will be studied in this section. The first 
problem is: Given the values of a Boolean function, how can a Boolean expression 
that represents this function be found? This problem will be solved by showing that 
any Boolean function may be represented by a Boolean sum of Boolean products of the 
variables and their complements. The solution of this problem shows that every Boolean 
function can be represented using the three Boolean operators - +, and The second 
problem is: Is there a smaller set of operators that can be used to represent all Boolean 
functions? We will answer this problem by showing that ail Boolean functions can be 
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EXAMPLE l 


TABLF 1 

X 

y 

z 

F 

G 

1 

i 

1 

0 

0 

1 

i 

0 

0 

1 

1 

0 

1 

! 

0 

I 

0 

0 

0 

0 

0 

1 

l 

0 

0 

0 

1 

0 

0 

1 

0 

0 

i 

0 

0 

0 

0 

0 

0 

0 


represented using only one operator. Both of these problems have practical importance 
in circuit design. 


SUM-OF-PRODUCTS EXPANSIONS 


We will use examples to illustrate one important way to find a Boolean expression that 
represents a Boolean function. 


Find Boolean expressions that represent the functions Fix, y, z) and G(x, y, z), which 
are given in Table i. 

Solution: An expression that has the value 1 when x = z - 1 and y = (1, and the value 
0 otherwise, is needed to represent F. Such an expression can be formed by taking 
the Boolean product of x, y, and z. This product, xyz, has the value 1 if and only if 
x = y - z = 1, which bolds if and only if x ~ ; = 1 and y - 0. 

To represent G, w r e need an expression that equals 1 when x ~ y — 1 and z - 0, 
or when jt = j = Oandy-l. We can form an expression with these values by taking 
the Boolean sum of two different Boolean products. The Boolean product xyz has the 
value 1 if and only if x = y - I and - = 0. Similarly, the product xyz has the value 1 
if and only if v = z = Gandy = 1. The Boolean sum of these two products, xyz+xyz, 
represents G, since it has the value 1 if and only if.r = v = 1 andz = Oorx = z = 0 
and v = I . > 


Example 1 illustrates a procedure for constructing a Boolean expression represent¬ 
ing a function with given values. Each combination of values of the variables for which 
the function has the value 1 leads to a Boolean product of the variables or their comple¬ 
ments. 

DEFINITION 1 . A literal is a Boolean variable or its eomptemetit. A rnintem of 
the Boolean variables xi,x?,....x n is a Boolean product yjyj ••• y,, where yt = x t 
or y, = Hence, a mintenn is a product of n literals, with one literal for each 
variable. 
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EXAMPLE 2 


EXAMPLE 3 


A minlerm has the value 1 for one and only one combination of values of its variables. 
More precisely, the minterm Vj v ^ ■ y„ is 1 if and only if each v, is 1, and this occurs 

if and only if .r, ■= 1 when v, - .r, and x, -■ (1 when y, - .r,-. 


Find a ntmterm that equals 1 if r t ~ At 0 and x: t* - x<, I. and equals 0 

otherwise. 

.Solution: The minterm ,v; a 7 .V(.t 4 .t» has the correct set of values ■ 


By raking Boolean sums of distinct minterms we can huild up a Boolean expression 
with a specified set of values. In particular, a Boolean sum of minterms has the value 1 
when exactly one of the minterms in the sum has the value 1. It has the value 0 for all 
other combinations of values of the variables. Consequently, given a Boolean function, 
a Boolean sum of minterms can be formed that has the value 1 when this Boolean 
function has the value 1, and has the value 0 when the function has the value 0 The 
minterms in this Boolean sum correspond to those combinations of values for which the 
function has the value 1, The sum of minterms that represents the function is called the 
sum-of-products expansion or the disjunctive normal form of the Boolean function. 


Find the sum-of-products expansion for the function fix, y, c) = (x y)r. 

Solution: The first step is to lind the values of F, These are found in Table 2. The sum- 
of-products expansion of F is the. Boolean sum of three minterms corresponding to the 
three rows of this table that give the value 1 for the function, This gives 

fix, y. z) = -tya + xy: •*- xyr. ■ 


It is also possible to find a Boolean expression that represents a Boolean function 
by taking a Boolean product of Boolean sums. The resulting expansion is called the 
conjunctive normal form or product-of-sums expansion of the function. These ex¬ 
pansions can be found from sum-of-products expansions by taking duals. How to find 
such expansions directly is described in Exercise 10 at the end of this section. 


TABLE 

n 




x ] y 1 

z 

.t+.V 

2 

(x i- y)z 

1 i 1 

1 

] 

0 

0 

Mil; 

0 

I 

1 

I 

1 i 0 

1 

' 

0 

0 

1 ! til 

0 

i 

1 

1 

0 ' 1 ! 

1 

1 

0 

0 

0 1! 

0 

1 

1 

l 

0 ! 0 1 

1 

0 

0 

0 

0 0 

0 

0 

1 

0 

.... 
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FUNCTIONAL COMPLETENESS 

Every Boolean function can. be expressed as a Boolean sum of minterms. Each rruntenn 
is the Boolean product of Boolean variables or their complements. This shows that every 
Boolean function can be. represented using the Boolean operators +, and ' . Since 
ever/ Boolean function can be represented using these operators we say that the set 
{■,+.“} is functionally complete. Can we find a smaller set of functionally complete 
operators? We can do so if one of the three operators of this set can be expressed in terms 
of the other two. This can be done using one of De Morgan’s laws. We can eliminate 
all Boolean sums using the identity 

x - y = x y, 

w hich is obtained by taking complements of both sides in the second De Morgan’s law, 
given in Tabic 5 in Section 9.1, and then applying the double complementation law. 
This means that the set } is functionally complete. Similarly, we could eliminate all 
Boolean products using the identity 

xy - x + y. 

which is obtained by taking complements of both sides in the first De Morgan's law, 
given in Table 5 in Section 9.1, and then applying the double complementation law. 
Consequently {+,'} is functionally complete. Note that the set {+, ■} is not functionally 
complete, since it is impossible to express the Boolean function F(x) = x using these 
operators (see Exercise 19). 

We have found sets containing two operators that are functionally complete. Can 
we find a smaller set of functionally complete operators, namely, a set containing just 
one operator? Such sets exist. Define two operators, the | or NANI) operator, defined 
by 1-1=0 and 1 j 0 = 0 | 1 = 0'0 - 1: and the j or NOR operator, defined by 
I j 1 = / i 0 - 0 I 1 = 0 and 0 ; 0 - 1. Both of (he sets {f} and {j .} are functionally 
complete. To see that {i} is functionally complete, since " } is functionally complete, 
all that we have to do is show that both of the operators • and ” can be expressed using 
just the ' operator. This can he done us follows: 

i - x \ x. 

xy ~ (x ' v) | (x | y). 

The reader should verify these identities (see Exercise 14). We leave the demonstration 
that { l } is functionally complete for the reader (see Exercises 15 and 16). 


Exercises 

1, Find a Boolean product of the Boolean van allies i, v, 
and;, or their complements, that has the value I if and 
only if 

a) .v =■ y = 0,: = ! b) a - 0. y = U = 0 

c) x = 0, y - ; - 1 d) v = y = ; = 0 

2, Find the sum-of-products expansions of the following 
Boolean functions. 

a) Fix, y) =■ t + y b) F(x,y) -- xy 
c) F(x,y) I d) F(x.y) = y 

3, Find the sum-of-products expansions of the following 
Boolean functions. 


a) /'(.v, y, i) = .v + y + z 

b) Fix, v. z) = (x + z)y 

c) Fix, y, z) = x 
di F(x, y, z) = x y 

4. Find the sum-of-products expansions of ilie Boolean 
function F(x, y, z) that equals 1 if and only if 

a) x - 0. b) xy - 0. 

c) A - v - 0. d) a y z = 0. 

5. Find the sum-of-products expansion of the Boolean 
function F(w, x, y.z) that has the value 1 if and only 
if an odd number of w, _i, y. and z have the value 1. 
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6. Find (he sum-oT-products expansion of the Boolean 
function F { 11 . as. r i, ax, x s ! that has the value 1 if and 
only if three or more of the variables >i, x% xj, sx. and 
(, have the value 1. 

Another way to find a Boolean expression that represents a 

Boolean function is to form a Boolean product of Boolean 

sums of literals. Exercises 7-11 are concerned with repre¬ 
sentations of this kind. 

7. Find a Boolean sum containing either x or x, either v or 

v, and either; or g that has the value 0 if and only if 
a) v — y ~ l,g = 0, b) x = y = = 0. 

c) r - ; - 0 v - 1. 

8. Find a Boolean product of Boolean sums of literals that 
has the value 0 if and only if either a = y = land" - 
0 , a -- : ~ 0 and v = 1, or x = y = : = 0. [Him; 
Take the Boolean product of the Boolean sums found 
in parts (a), (b), and (c) in Exercise 7,] 

9. Show that the Boolean sum Vi - yi f ■ • ■ — y„, where 
v, - or V/ — a,, has the value 0 for exactly one com¬ 
bination 0 )' the values of the variables, namely, when 
x, - Oify, = X; and x, = 1 if v, = xj, This Boolean 
sum is called a maxterm. 

10, Show that a Boolean function can be represented as a 
Boolean product of maxtersm. This representation is 
culled the product-of-sums expansion or conjunc¬ 
tive normal form of the function. (Hint: Include one 
maxterm in this product for each combination of the 

93 __ 

Logic Gates 

INTRODUCTION 


Boolean algebra is used to model the circuitry' of electronic devices. Each input and each 
Web ou! P ut ° f suc h a device can be thought of as a member of the set {(), 1}. A computer, or 
other electronic device, is made upof a number of circuits. Each circuit can be designed 
using the rules of Boolean algebra that were studied in Sections 9.1 and 9.2. The basic 
elements of circuits are called gates. Each type of gate implements a Boolean operation. 
In this section we define several types of gates. Using these gates, we will apply the rules 
of Boolean algebra to design circuits that perform a variety of tasks. The circuits that 
we will study in this chapter give output that depends only on the input, and not on the 
current state of the circuit. In other words, these circuits have no memory' capabilities. 
Such circuits are called combinational circuits or gating networks. 

We will construct combinational circuits using three types Of elements. The hrst 
is an inverter, w hich accepts the value of one Boolean variable as input and produces 
the complement of this value as its output. The symbol used for an inverter is shown 
in Figure 1(a). The input to the inverter is shown on the left side entering the element, 
and the output is shown on the right side leaving the element. 

The next type of element we will use is the OR gate. The inputs to this gate are the 
values of two or more Boolean variables. The output is the Boolean sum of their values. 


variables where the function has the value 0.) 

11. Find the product-of-sums expansion of each of the 
Boolean functions in Exercise 3. 

12. Express each of the following Boolean functions using 
the operators ■ and ", 

a) x -T v 4 z b) x 4- v (a 4 c) 

c) [x 4 y) d) i(x 4 y 4 cl 

13. Express each of the Boolean functions in Exercise 12 
using the operators r and “. 

14. Show that 

a) x = x j a, 

b) xy = ! a | >. 1 1 (,v ■ r), 

c) x 4 v - (a ; x ), (y | v). 

15. Show that 

a) x = x i x. 

b) xy - (x l n J (y j y), 

c) x 4 v = (a 1 yl I (x l y). 

16. Show that { j } is functionally complete using Exer¬ 
cise 15. 

17. Express each of the Boolean functions in Exercise 3 
using the operator j. 

18. Express each of the Boolean functions in Exercise .3 
using the operator 1, 

19. Show' that the set of operators { 4 , ■} is not functionally 
complete. 

20. Are the following sets of operators functionally com- 
plete? 

a) {4,*} b}{-.*} cM-.ffll 
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FIGURE 1 Basic Types of Gates. 



FIGURE 2 Cates with n Inputs, 


The symbol used for an OR gate is shown in Figure 1(b), The inputs to the OK gate are 
shown on the left side entering the element, and the output is shown on the right ■side 
leaving the element. 

The third type of element We w ill use is the AND gate. The inputs to this gate are 
the values of two or more Boolean variables. The output is the Boolean product of their 
values. The symbol used for an AND gate is shown in Figure 1(c). The inputs to the 
AND gate are shown on the left side entering the element, and the ouEput is shown on 
the right side leaving the element. 

We will permit multiple inputs to AND and OR gates. The inputs to each of these 
gates are shown on the left side entering the dement, and the output is shown on the 
right side. Examples of AND and OR gates with n inputs are shown in Figure 2. 


COMBINATIONS OF GATES 


Combinational circuits can be constructed using a combination of inverters, OR gate, 
and AND gates. When combinations of circuits are formed, some gates may share in¬ 
puts. This is shown in one of two ways in depictions of circuits. One method is to use 
branchings that indicate all the gates that use a given input. The other method is to in¬ 
dicate this input separately for each gate. Figure3 illustrates the two ways of showing 




FIGURE 3 Two Ways to Draw the Same Circuit. 
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FIGURE 4 Circuits That Produce the Outputs Specified in Example I. 


gates with the same input values. Note also that output from agate may be used as input 
by one or more other elements, as show in Figure 3. Both drawings in Figure 3 depict 
the circuit that produces the output xy +- xy. 


Construct circuits that produce the following outputs: (a) {x 4 y)j, (b) x(v 4 z), and 

[Cl (X i- V 4 Z)(-t V ’). 

Solution: Circuits that produce these outputs are shown in Figure 4. ■ 


EXAMPLES OF CIRCUITS 

We will give some examples of circuits that perform some useful functions. 
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FIGURE 5 A Circuit for Majority Voting, 


EXAMPLE 2 A committee of three individuals decides issues for an organization. Each individual 

vuies either yes or no for each proposal that arises. A proposal is passed if it receives at 
least two yes votes. Design a circuit that determines whether a proposal passes. 

Solution: Let x = 1 if the first individual votes yes, and a- = 0 if this individual votes 
no; let y - 1 if the second individual votes yes, and y = 0 if this individual votes no; 
let - = I if the third individual votes yes. and z ~ 0 if this individual votes no. Then 
a circuit must be designed that produces the output 1 from the inputs x, y, and z when 
two or more of x, y, and z are I, One representation of the Boolean function that has 
these output values is jcv + xz + yz (see Exercise 6 in Section 9.1). The circuit that 
implements this function is shown in Figure 5. ■ 


EXAMPLE 3 Sometimes light fixtures are controlled by more than one switch. Circuits need to be 

designed so that flipping any one of the switches for the fixture turns the light on when 
it is off and turns the light off when it is on. Design circuits that accomplish this when 
there are two switches and when there are three switches. 


Solution: We will begin by designing the circuit that controls the light fixture when two 
different switches are used. Let x = 1 when the first switch is closed and x = 0 when 
it is open, and let y = i when the second switch is closed and y = 0 when it is open. 
Lei Fix. y) = 1 when the light is on and F(x, v) = 0 when it is off. We can arbitrarily 
decide that the light will be on when both switches are dosed, so that /■'( I, I) — 1. This 
determines all the other values of F. When one of the two switches is opened, the light 
goes off, so Ell, 0) = F(0, 1) = 0. When the other switch is also opened, the light goes 
on, so that F(0, 0) = l. Table 1 displays these values. We see that F(x, y) — xy + x v. 
This function is implemented by the circuit shown in Figure 6. 

We will now design a circuit for three switches. Let x, y, and z be the Boolean 
variables that indicate whether each of the three switches is closed. Wc let x = I when 
the first switch is closed, and x = 0 when it is open; y = 1 when the second switch 
is closed, and y - 0 when it is open; and z — 1 when the third switch is closed, and 


TABLE 1 

X 

y 

F{ %j) 

1 

] 

i 

0 

1 

0 

a 

1 

0 

0 

0 

1 



FIGURE 6 A Circuit for a Light Controlled by Two Switches. 


PDF created with pdfFactory trial version www.pdffactorv.com 





























') / Itoalean Algebra 


608 


tabu: 2 

X 

9 \ z 

Ffx.y, z) 

i 

i 

! 

i ' 1 

i ; » 

i 

0 

i 

o i I 

0 

i 

0 0 

1 

o 

i ! 1 

0 

0 

1 1 0 

1 

0 

0 ' I 

1 

0 

0 i 0 

0 


C = 0 when il is open. Let Fix, y, :J - ! when the light is on and Fix, v, z) = 0 when 
the light is off. We can arbitrarily specify that the light be on when all three switches are 
closed so that F(l, 1, i) = 1. Tins determines all other values of F. When one switch 
is opened, the light goes off so that F[l, 1,0) = F(l,0, 1) = F( 0, 1, 1) - 0. When a 
second switch is opened, the light goes on so that F{ 1,0, 0} = F(0. 1,0) = F(Q,0,1) - 
1. Finally, when the third switch is opened, the light goes off again so that F(0,0,0) = 
0. Table 2 shows the values of this function. 

The function F can be represented by its sum-of-products expansion so that 
Ftx.y.z) ■*- xyz + jtjrz ■+■ xyz + xyz • The circuit shown in Figure 7 implements 
this function. B 



f it j I iRF, 7 A Circuit for a Fixture (lontrolled by Three Switches. 
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ADDERS 

We will illustrate how logic circuits can be used to carry out addition of two positive 
integers front their binary expansions. We will build up the circuitry to do this addition 
from some component circuits. First, we will build a circuit that can be used to find 
t + v, where x and y are two bits. The input to our circuit will be x and y, since these 
each have the value 0 or the value 1. The output will consist of two bits, namely, s and 
C, w here s is the sum bit and c is the carry bit. This circuit is called a multiple output 
circuit since it has more than one output. The circuit that we are designing is called the 
half adder, since it adds two bits, without considering a carry from a previous addition. 
We show the input and output for the half adder in Table 3. From Table 3 we see that 
c = jry and that s = ay + xy - (.v + y)(,xy). Hence, the circuit shown in Figure K 
computes the sum bit s and the carry bit c from the bits x and y. 

We use the full adder to compute the sum bit and the carry' bit when two bits 
and a carry are added. The inputs to the full adder are the bits x and y and the carry 
r,. The outputs are the sum bit s and the new carry ry 4 1 . The inputs and outputs for the 
full adder are shown in Table 4. 


TAB I K 4 Input and 
Output fur the Full 
Adder. 

Input 

Output 

X 

y 

Ci 

S 

0 . ! 

] 

1 

1“ 

i 

i 

t 

0 

i 

0 

1 

1 

1 

0 

1 

0 

1 

1 

0 

0 

1 

0 

0 

1 

1 

0 

1 

0 

1 

0 

1 

0 

0 

0 

1 

l 

0 

(1 

0 

0 

0 

n 


tabif 3 

Input and 
Output for the 
Hall Adder. 

Input 

Output 

X V 

s 

C 

1 1 

0 

l 

1 0 

i 

0 

o : 1 

i 

0 

(1 0 
_ : _ 

0 

0 
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l f(.l K! l < 4 Full Adder. 




fllil RF II) Adding Two Three-Bit Inte¬ 
gers with Full and Half Adders. 


I he twn outputs of the full adder, the sum bit s and the carry v,., ,, are given by the 
sum-of-products expansions xye, r .vyr. ■“ x\r, -r xyCf and xyc, i .nr, + a it, +,tvt-„ 
respectively. However, instead of designing the full adder from scratch, we will use half 
adders to produce the desired output. A full adder circuit using half adders ts shown in 
figure 

finally, in Figure ] I) we show how lull and half adders can be used to add the two 
three-bit integers ugAi-Vob and (ygyi yo): to produce the .sum (ij.52.Si.tnb- Note that ip. 
the highest-order bit in the sum, is given by the carry tw. 


Exercises 

hi Exercises I -5 find the rnilpi.it of the given circuit. 

1 ■ . ► \ _ 

•- 

HH '\ 

.i-► 

-W’>:- 

2 ^ 

■ - ,X>- 

■ — ► 

-K" 

► ► \j 

r.* 
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6. Construct circuits from inverters, AND gates, and OR 
gates to produce the following outputs, 

a I a + v ht (a f >').v 

cl ay; i j i - d) (a + r)(v z) 

7. Design ti circuit that implements majority voting for 
live individuals. 

8. Design u circuit for a light fixture controlled by four 
switches where flipping one of the switches turns the 
light on when it is off and turns it off when it is on. 

9. Show how 1 the sum of two five-bit integers can be found 
using ful l and half adders. 

ill. Construct a circuit for a half subtracter using AND 
gaicx OH gates, and inverters. A half subtracter has 
I vv i bits as input und produces ns output a difference 
bit and a borrow. 

11. Construct a circuit for a full subtracter using AND 
gates, OR gates, and inverters. A full subtracter has 
two bits and a borrow' as input, and produces as output 
a difference bit and a borrows 

12. 1. ■ se the circuits from Exercises 10 and 11 to find the 
difference of two Ibur-bit integers, where the first inte¬ 
ger is greater than the second integer. 

*13, Construct a circuit that compares the two-bit integers 
(a|- iv,b and (vjtyJK, returning an output of ] when 
the first of these numbers is larger and an output of 0 
otherwise. 


*14, Construct a circuit that computes the product of the 
two-bit integers (a , a u >; and (j] y C( ) 2 . The circuit should 
have four output bits for the bits in the product. 

Two gates that arc often used in circuits are NAND and NOR 
gates. When NANI) or NOR gates are used to represent cir¬ 
cuits, do other types of gates are needed. The notation for 
these gates is ns follows: 



*15. Lise NAND gates to construct circuits with the follow¬ 
ing outputs. 

a) r t>) x t y e) a v d) x&y 
*16. Use NOR gales to construct circuits for the outputs 
given in Exercise !5. 

*17. Construct a half adder using NAND gates, 

*18. Construct a half adder using NOR gates. 

A multiplexer is a switching circuit that produces as output 
one of a set of input bits based on the value of control bits. 

19. Construct a multiplexer using AND gates, OR gates, 
and inverters that has as input the four bits r 0 , a,, x~., 
and a, and the two control bits <r u and a , Set up the 
circuit so that x f is the output where i is the value of 
the two-bit integer (a Cola - 


9.4 __ 

Minimization of Circuits 

INTRODUCTION 


The efficiency of a combinational circuit depends on the number and arrangement of 
its gates. The process of designing a combinational circuit begins with the table spec¬ 
ifying the output for each combination of input values. We can always use the sum- 
of-products expansion of a circuit to find a set of logic gates that will implement this 
circuit. However, the sum-of-products expansion may contain many more terms than 
arc necessary. Terms in a sum-of-products expansion that differ in just one variable, 
so that in one term this variable occurs and in the other term the complement of this 
variable occurs, can be combined. Far instance, consider the circuit that has output l if 
and only if x = y - z = 1 or x = z = 1 and y = 0. The sum-of-products expansion 
of this circuit is xyz + xyz. Tire two products in this expansion differ in exactly one 
variable, namely, y. They can be combined as follows: 

-i'y: + .tvr = (y + v)(.v;) 

= 1 - (x-> 

= xz 

Hence, xz is a Boolean expression with fewer operators that represents the circuit. We 
show two different implementations of this circuit in Figure L The second circuit uses 
only one gate, whereas the first circuit uses three gates and an inverter. 
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HGURE1 Two Circuits with the Same Output, 


This example shows that combining terms in (he sumof-products expansion of a 
circuit leads to a simpler expression for the circuit. We will describe two procedures 
that simplify sum-of-products expansions. The goal of both of these procedures is to 
produce Boolean sums of Boolean products that contain the least number of products 
of literals such that these products contain the least number of literals possible among 
all sums of products that represent a Boolean function. 

The techniques described in this section for simplifying sum-of-product expansions 
are still of practical value. However, modem circuits are often built using more compli¬ 
cated types of elements than AND gates, OR gates, and inverters. Various procedures 
are used to simplify circuits built using these more complicated dements. However, 
many of these methods use ideas similar to those described in this section. 


KARNAUGH MAPS 


To reduce the number of terms in a Boolean expression representing a circuit, it 
wgfo is necessary to find terms to combine. There is a graphical method, called a Kar¬ 
naugh map, for finding terms to combine for Boolean functions involving a relatively 
small number of variables. The method we will describe was introduced by Mau¬ 
rice Karnaugh in ld53. His method is based on earlier work by E. W. Veitch. (This 
method is usually applied only when the function involves six or fewer variables.) 
Karnaugh maps give us a visual method for simplifying sum-of-products expan¬ 
sions; they are not suited for mechanizing this process. We will first illustrate how Kar¬ 
naugh maps are used to simplify expansions of Boolean functions in two variables . 

There are four possible minterms in the sum-of-products expansion of a Boolean 
function in the two variables jc and y, A Karnaugh map for a Boolean function in these 
two variables consists of four squares, where a 1 is placed in the square representing 


Maurice Karnaugh Itiorn 1424 I. Maurice Karnaugh, bunt in New York City, received ha. B .S. from the 
City College of New York and his MS. and Ph.lX from Yale University. He was a member of the technical 
staff at Bel! Laboratories from 1952 until 1966 and Manager of Research and Development at the federal 
Systems Division of AT&T from 1966 to 1970. In 1970 he joined IBM as a member of the research staff. 
Karnaugh has matte fundamental contributions to the application of digital techniques in both computing 
and telecommunications. His current interests include knowledge-based systems in computers and heuristic 
NCai L'h meth(xiv 
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EXAMPLE 1 


EXAMPLE 2 



F1GI RE 2 Karnaugh Maps in Two Variables. 


a minterm if this minterm is present in the expansion. Squares are said to be adjacent 
if the minterms that they represent differ in exactly one literal. For instance the square 
representing xy is adjacent to the squares representing ,vv and xy, The four squares 
and the terms that they represent are shown in Figure 2. 


Find the Karnaugh maps for (a) xy +- xy, (b) xy + xy, and (c) xy + xy + x v. 

Solution; We include a 1 in a square when the mintenn represented by this square 
is present in the sum-of-produets expansion. 'Hie three Karnaugh maps tire shown in 
Figure 3. ■ 


We can identify minterms that can be combined from the Karnaugh map. When¬ 
ever there are Is in two adjacent squares in the Karnaugh map. the minterms rep¬ 
resented by these squares can be combined into a product involving just one of the 
variables. For instance, xy and xy are represented by adjacent squares and can be 
combined into v, since xy + xy ~ (x + x)y - y. Moreover, if Is are in all four 
squares, the four minterms can be combined into one term, namely, the Boolean ex¬ 
pression 1 that involves none of the variables. We circle blocks of squares in the 
Karnaugh map that represent mmterms that can be combined and then find the corre¬ 
sponding sum of products. The goal is to identify the largest possible blocks, and to 
cover all the Is with the fewest blocks using the largest blocks first and always using 
the largest possible blocks. 


Simplify the sum-of-product? expansions given in Example !. 

Solution: The grouping of minterms is shown in Figure 4 using the Karnaugh maps for 
these expansions. Minimal expansions for these sums-of-products are (a}_v, (b) xy +- xy, 
and (c) x + y. ■ 



(» (M le) 

FIGURE 3 Karnaugh Maps for the Sum-of-Products Expansions 
in Example 1, 
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EXAMPLE 3 



/" "v 

; i 1 

'v y 

0 



/"~"N 
? i ' 


i 



l ill 


(bi 


let 


FIGURE 4 Simplifying the Sum-of-Products Expansion from Example 1. 


A Karnaugh map in three variables is a rectangle divided into eight squares. The 
squares represent the eight possible minterms in three variables. Two squares are said 
to be adjacent if the minterms that they represent differ in exactly one literal. One of the 
ways to form a Karnaugh map in three variables is shown in Figure 5(a), This Karnaugh 
map can be thought of as lying on a cylinder, as shown in Figure 5(b). On the cylinder 
two squares have a common border if and only if they are adjacent. 

To simplify a sunroof-products expansion in three variables, we use the Karnaugh 
map to identify blocks of minterms that can be combined. Blocks of two adjacent 
squares represent pairs of mintenns that can be combined into a product of two lit¬ 
erals; 2x2 and 4 x 1 blocks of squares represent minterms that can be combined into 
a single literal; and the block of all eight squares represents a product of no literals, 
namely, the function 1. In Figure 6,1 x 2,2 x 1, 2 x 2,4 x 1, and 4 x 2 blocks and the 
products they represent are shown. 

The goal ts to identify the largest possible blocks in the map and cover all the 1 s 
in the map with the least number of blocks, using the largest blocks first. The largest 
possible blocks are always chosen. Note that there may be more than one way to do this. 
The following example illustrates how Karnaugh maps in three variables are used. 


Use Karnaugh maps to simplify the sum-of-products expansions (a) xyz + xy z + xyz 4 
x y t (b) xyz-F xy Irlyz+ x yz+x y z, and (c) xyz+ xyz-xyz + xy z+xyz+x yz+xy z. 

Solution: The Karnaugh maps for these sum-of-products expansions are shown in 
Figure 7. The grouping of blocks shows that minimal expansions into Boolean sums of 
Boolean products are (a) xz + y z + Ayr, (b) v - Az. and (c) x + y -r z- ■ 



Uj 

FIGURE 5 Karnaugh Maps in Three Variables, 
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r = at; - xyz +■ -tv; - x\ ■ 
(e) 


if v; ;>r u w it 



(d) (e) 

FIGL RE 6 Blocks in Karnaugh Maps in Three Variables, 


A Karnaugh map in four variables is a square that is divided into 16 squares. The 
squares represent the 16 possible minterms in four variables. One of the ways to form 
a Karnaugh map in four variables is shown in Figure 8. 

Two squares are adjacent if and only if the minterms they represent differ in one 
literal. Consequently, each square is adjacent to four other squares. The Karnaugh map 
of a sum-of-products expansion in four variables can be thought of as lying on a torus, so 
that adjacent squares have a common boundary (see Exercise 20). The simplification of 
a sum-of-products expansion in four variables is carried out by identifying those blocks 
of 2, 4, 8, or 16 squares that represent minterms that can be combined. Each square 
representing a minterm must either be used to form a product using fewer literals, or be 
included in the expansion. In Figure 9 some examples of blocks that represent products 
of three literals, products of two literals, and a single literal are illustrated. 

As is the case in Karnaugh maps in two and three variables, the goal is to identify 
the largest blocks of 1 s in the map and to cover all the 1 s using the fewest blocks needed, 
using the largest blocks first. The largest possible blocks are always used. The following 
example illustrates how Karnaugh maps in four variables are used. 


EXAMPLE 4 Use Karnaugh maps to simplify the sum-of-products expansions (a) n-v v. -t wxyz f 

wxy z + w'iyz + wxyz + wxy-z + wxyz + w tyz + w xyz, (b) wxy z + w.tvc + nety; f 
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FIGURE 7 Using Karnaugh Maps in Three Variables. 
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y- yz K v; 



FIGURE 8 Karnaugh Map in Four Variables. 


vv.Yyr + wry: + wjryz + wjty z, and (c) H’.tvz + wxjl + wxyz + H’jtyz + wxyz + 
wxyz + wxyz * vvjtyI + wxyz + wxyz + very:. 

Solution: The Karnaugh maps for these expansions are shown in Figure 10. Using 
the blocks shown leads to the sum of products (a) vi y- + wxz + wxy + wxy + wry :, 
(b) y: + w.Vv + xyz, and (c) : + ra + wry. The reader should determine whether 


WX 


JV.l 


u xz = ft 'xy: + n'.ryc 


(a) 



vt-fv- 4- v,xyr 




ua 


MX: 

wx - wxyz + wxy® - 
S'jy: + g jfyc 

0>J 


y:. v:. v; 



















: = iv; + hmv." + wxy: + 
M'xyz + u .i+ iva v: + h'.i v“ * iT'.f y, 


tel id) 

FIGURE 9 Blocks in Karnaugh Maps in Four Variables. 
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FIGURE 10 Using Karnaugh Maps in Four Variables, 



there are other choices of blocks in each part that lead to different sums of products 
representing these Boolean functions. ■ 


DON'T CARE CONDITIONS 


In some circuits we care only about the output for some combinations of input values. 

W#b since the other combinations of input values never arise. This gives us freedom in pro¬ 
ducing a simple circuit with the desired output since the output values for all those 
combinations that never occur can be arbitrarily chosen. The values of the function for 
these combinations are called don't care conditions, A d is used in a Karnaugh map 
to mark those combinations of values of the variables for which the function can be 
arbitrarily assigned. In the simplification process we can assign Is as values to those 
combinations of the input values that lead to the largest blocks in the Karnaugh map. 
This is illustrated in the following example. 


EXAMPLE 5 One way to code decimal expansions using bits is to use the four bits of the binary 

expansion of each digit in the decimal expansion. For instance, 873 is encoded as 
100001110011. This encoding of a decimal expansion is called a binary coded deci¬ 
mal expansion, Since there are 16 blocks of four bits and only 10 decimal digits, there 
are six combinations of four bits that are not used to encode digits. Suppose that a cir¬ 
cuit is to be built that produces an output of 1 if the decimal digit is 5 or greater and an 
output of 0 if the decimal digit is less than 5. How can this circuit be simply built using 
OR gates, AND gates, and inverters? 

Solution: Let F(u>, x, y. z) denote the output of the circuit, where wxyz is a binary 
expansion of a decimal digit. The values of F are shown in Table I. The Karnaugh 
map for F, with d$ in the don’t care positions, is shown in Figure 11(a). We can ei¬ 
ther include or exclude squares with ds from blocks. This gives us many possible 
choices for the blocks. For example, excluding all squares with rfs and forming blocks 
as shown in Figure 11(b) produces the expression wr y + wxy + wxz. Including some 
of the c/s and excluding others and forming blocks as shown in Figure 11(c) produces 
the expression wx + wxy + xyz- Finally, including all the ds and using the blocks 
shown in Figure 1 ltd) produces the simplest expansion possible, namely, F(jc, y,;) - 
w 4- vy 4- ■ 
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1 

TABLE 1 

Digit 


X 

V 

z 

F 

0 

fj 

C 

0 

0 

0 

1 

0 

0 

0 

1 

0 
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0 

0 

i 

0 

t) 

3 

0 

0 

1 

1 

0 

4 

a 

1 

(I 
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0 

5 

0 

1 

0 

1 

1 i 

6 

0 

1 

I 

0 

1 

7 

it 

] 

1 

! 

1 

8 

i 

0 

(i 

0 

1 

9 

i 

0 

0 

1 

1 



FIGURE 11 The Karnaugh Map for F Showing its Don’t Care Positions. 
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THE QUINE-McCLUSKEY METHOD 


We have seen that Karnaugh maps can be used to produce minimal expansions of 
H Boolean functions as Boolean sums of Boolean products. However, Karnaugh maps 
are awkward to use when there are more than four variables. Furthermore, the use of 
Karnaugh maps relies on visual inspection to identify terms to group. For these reasons 
there is a need for a procedure for simplifying sum-of-products expansions that can be 
mechanized. The Quine-McCluskey method is such a procedure. It can be used for 
Boolean functions in any number of variables. It was developed in the 1950s by W. V. 
Quine and E. J. McCluskey, Jr. Basically, the Quine-McCluskey method consists of 
two parts. The first part finds those terms that are candidates for inclusion in a minimal 
expansion as a Boolean sum of Boolean products. The second part determines which of 
these terms to actually use. We will show how this procedure works using an example. 

EXAMPLE 6 We will show how the Quine-McCluskey method can be used to find a minimal expan¬ 

sion equivalent to 

xyz + xyz - 1 xyz -r x y~ + x y t 

We will represent the minterms in tins expansion by bit strings. The first bit will be 
1 if a occurs and 0 if x occurs. The second bit will be 1 if y occurs and 0 if y occurs. The 
third bit will be i if z occurs and 0 if j occurs. We then group these terms according to 
the number of Is in the corresponding bit strings. This information is shown in Table 2. 

Minterms that can be combined are those that differ in exactly one literal. Hence, 
two terms that can be combined differ by exactly one in the number of Is in the bit 
strings that represent them. When two minterms are combined into a product, this prod¬ 
uct contains two literals. A product in two literals is represented using a dash to denote 


Edward .J. McCluskey (born 1929). Edward McCluskey attended Bowdoin College and M.I.T., where 
he received his doctorate in Electrical Engineering in 1956. Rejoined Bell Telephone Laboratories in 1955. 
remaining there until 1959. McCluskey was professor of Electrical Engineering at Princeton University from 
1959 until 1966, also serving as Director of the Computer Center at Princeton from 1961 to 1966. In 1967 he 
took a position as professor of Computer Science and Electrical Engineering at Stanford University, where 
he also served as director of the Digital Systems Laboratory from 1969 to 1978 McCluskey has worked in 
a variety of areas in computer science, including fault-tolerant computing, computer archi tec tore, testing, 
and logic design. He is currently director of the Center fur Reliable Computing at Stanford University. 
McCluskey is also an ACM Fellow. 

Willard Van Orman Quine (born 1908}. Willard Quine, bom in Akron, Ohio, attended Oberlin College 
and later Harvard University, where lie received his Ph.D. in Philosophy in 1932, He became a Junior 
Fellow at Harvard in 1933 and was appointed to a position on the faculty there in 1936. He has remained at 
Harvard his entire professional life, except for World War II, when he worked for the ILLS, Navy decrypting 
messages from Gentian submarines. Quine has always been interested in algorithms, but not m hardware, 
He arrived at his discovery of what is now called the Quine-McCluskey method as a device far teaching 
mathematical logic, rather than as a method for simplify ing switching circuits. Quine is one of the most 
famous philosophers in the world. He has made fundamental contributions to the theory of knowledge, 
mathematical logic and set theory, and the philosophies of logic and language, His books including kw 
Foundations of Matkemat&al Logic published in 1937 and Word and Object published in I960, have had 
profound impact. Quine retired from Harvard in 1978 but continues to commute from his home in Beacon 
Hill to his office there. He still uses the 1927 Remington typewriter on which he prepared his doctoral thesis. 
Kmg ago he had an operation performed on this machine to add a few special symbols, removing the second 
period, the second comma, and the question mark. When asked whether he missed the question mark, he 
replied, “Well, you see, 1 deal in certainties.” There is even a word quine, defined in the Now Hackee's 
Dictionary as a program that generates a copy of ils own source code as its complete output. Producing the 
shortest possible quine in a given programming language is a popular puzzle for hackers. 
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TABLE2 

Minterm 

Bit String 

Number of Is 

xyz 

111 

3 

xy; 

1(11 

2 

xyz 

Oil 

2 

xyz 

ooi ! i 

xyz 

000 

0 


the variable that does not occur For instance, the inimcims x v" and x yz, represented by 
bit strings 101 and (XI1, can be combined into yz, represented by the string -01. All pairs 
of niinterms that car, be combined and the product formed from these combinations are 
shown in Table 2, 

Next, all pairs of products of two literals that can be combined are combined into 
one literal. Two such products can be combined if they contain literals for the same two 
variables, and literals for only one of the two variables differ. In terms of the strings 
representing the products, these strings must have a dash in the same position and must 
differ in exactly one of the other two slots. We can combine the products yz and yc, 
represented by strings -11 and -01, into z, represented by the string —1. We show all 
the combinations of terms that can be formed in this way in Table 3, 

In Table 3 we also indicate which terms have been used to form products with 
fewer literals; these terms will not be needed in a minimal expansion. The next step is 
to identify a minimal set of products needed to represent the Boolean function. We begin 
with all tho se products that were not used to construct products with fewer literals. Next, 
we form Table 4, which has a row for each candidate product formed by combining 
original terms, and a column for each original term; and we put an X in a position if the 
original term in the sum-of-products expansion was used to form this candidate product. 
In this case, we say that the candidate product covers the original minterm, We need 
to include at least one product that covers each of the original min terms. Consequently, 
whenever there is only one X in a column in the table, the product corresponding to 


TABLE 3 

Step 1 

Step2 

Bit 

Term String Term String 

Term String 


l 

.tv; 

in 

| (1.2) 

xz 

i-i 

2 

xyz 

101 

; (1,3) 

yz 

-ii 

3 

xyz 

Oil 

| (2,4) 

yz 

-01 

4 

xyz 

(XJ1 

i CM) 

xz 

0-1 

5 

xyz 

000 

(4,5) 

xy 

00- 
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EXAMPLE 7 


TABLE 4 


xyz 

xyz 

xyz 

xyz 

ryz 

z 

X 

X 

X 

X 


xy 




X 

X 


the row this X is in must be used. From Table 4 we see that both z and x v are needed. 
Hence, the final answer is : + x y. B 


As was illustrated in Example 6, the Quine-McCluskey method uses the following 

sequence of steps to simplify a sum-of-products expression. 

1. Express each minterm in n variables by a bit string of length n with a 1 in the rth 
position if x, occurs and a 0 in this position if t, occurs. 

2. Group the bit strings according to the number of 1 s in them. 

3. Determine ail products in n - 1 variables that can be formed by taking the Boolean 
sum of minserms in the expansion. Mintemis that can be combined are represented 
by bit strings that differ in exactly one position. Represent these products inn - I 
variables with strings that have a 1 in the ith position if x, occurs in the product, a 0 
in this position if x, occurs, and a dash in this position if there is no literal involving 
Xt in the product. 

4. Determine al l products in n - 2 variables that can be formed by taking the Boo lean 
sum of the products in n - 1 variables found in the previous step. Products in n - \ 
variables that can be combined are represented by bit strings which have a dash in 
the same position and differ in exactly one position. 

5. Continue combining Boolean products into products in fewer variables as long as 
possible. 

6. Find all the Boolean products which arose that were not used to form a Boolean 
product in one fewer literal. 

7. Find the smallest set of these Boolean products so that the sum of these products 
represents the Boolean function. This is done by forming a table showing which 
minterms are covered by which products. Every minterm must be covered by at 
least one product, (This is the most difficult part of the procedure. It can be mech¬ 
anized using a backtracking procedure.) 

A final example will illustrate how this procedure is used to simplify asum-of-products 

expansion in four variables. 


Use the Quine-McCluskey method to simplify the sum-of-products expansion wxyz + 
h’.tvc + wxyz + wxyz + wxyz, + wxyz + wry;. 


Solution: We first represent the mintemis by bit strings and then group these terms 
together according to the number of Is in the bit strings. This is shown in Table 5. All 
the Boolean products that can be formed by taking Boolean sums of these products are 
shown in Table 6. 
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TABLE 5 

Term 

Bit String 

Number of Is 

wxyz 

1110 

3 

wxyz 

ion 

3 

wxyz 

out 

3 

Wxyz 

1010 

2 

wxyz 

0101 

2 

wxyz 

OOP 

2 

ZZIIi 

0001 

J_ 1 


TABLE 6 


Step i 

Step 2 



Bit 





Term 

S(nng 

Term 

String 

Term 

String 

1 ic.tyz 

mo 

! 1.4) wvz 

M0 

(3,5,6,7) Wz 

0 — ( 

2 wxyz 

1011 

(2,4) wxy 

101- 



3 wry; 

out 

(2,6) xyz 

AMI 



4 wxyz 

1010 

(3.5) wjcz 

01-1 



5 Wxyz 

0101 

(3,6) Wyz 

0-11 



6 Wxyz 

0011 

(5,7) Wyz 

0-01 



7 Wxyz 

0001 

(6,7) Wx; 

00-1 




The only products that were not used to form products in fewer variables are wz, 
wyz, wxy, and xyt In Table 7 we show the minterms covered by each of these products. 
To cover these minterms we must include wz and wyz, since these products are the only 
products that cover wxyz and wxyz, respectively. Once these two products are included, 
we see that only one of the two products left is needed. Consequently, we can take either 
wz + wyz + wxy or wz + wy> + xyz as the final answer. ■ 


TABLE 7 

1 wxyz 

wxyz 

Wxyz 

wxyz 

wxyz 

wxyz 

wxyz 

wz 



X 


X 

X 

X 

wyz 

X 



X 




wxy 


X 


X 




xyz \ 

X 




X 
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Exercises 


1. a) Draw a Karnaugh map fora function in two vari¬ 

ables and put a 1 in the square representing xy. 
b) What are the minterms represented by squares ad¬ 
jacent to this square? 

2, Find the siun-of-produtrs expansions represented by 
each of the following Karnaugh maps. 

a) b) 

y y 



3, Draw the Karnaugh maps of the following sum-of- 
products expansions in two variables. 

a) xy 

h i xy + ivy 

c) xy + xy + xy + xy 

4, Use a Karnaugh map to find a minimal expansion as 
a Boolean sum of Boolean products of each of the fol¬ 
lowing functions of the Boolean variables x and v. 

a) xy + x y 

b) xv + xy 

e) xy + xy + xy + x y 

5, a) Draw a Karnaugh map for a function in three vari¬ 

ables. Put a 1 in the square that represents xy: 
b) Which mintemis are represented by squares adja¬ 
cent to this square? 

6, Use Karnaugh maps to find simpler circuits with the 
same output as each of the following circuits shown 
below. 

a) 
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624 9 / Boolean Algebra 


7. Draw the Karnaugh maps of the following sum-of- 
products expansions in three variables. 

a) xv z 

b) v v.7 + x y z 

e) vv; + .tv; + xyz + xyz 

8. Use a Karnaugh map to find a minimal expansion 
as a Boolean sum of Boolean products of each of the 
following functions in the variables x, y, and ;. 

a) xyz t- xyz 

b) vv; + Xyz + r.K + xyi 

c) xyz r xyz + xyz +• xyz + xyz 

d) vv; + xyz + <v: + xyz + xyz + x y; 

9. a) Draw a Karnaugh map for a function in four vari¬ 

ables. Put a 1 in the square that represents wxyz. 

b) Which minterms are represented by squares adja¬ 
cent to this square? 

HI. Use a Karnaugh map to find a minimal expansion as 
a Boolean sum of Boolean products of each of the fol¬ 
lowing f unctions in the variables w. v, y, and;. 

a) ivrv; + wry; + wry ; + wxyz + nr y; 

b) u t yz + wry; + wxyz + wry; + very; + wry; 

c) wry; 4- wary; + wry; + wry; + wr y ; 4- wry; + 
wry; + wry; 

d) wrv; + w,«vi + wxyz + wry; + wry; + wry; + 
wv v; +■ wxyz + w xyz 

11. a) How many squares does a Karnaugh map in five 
variables have? 

b) How many squares are adjacent to a given square in 
a Karnaugh map in five variables? 

*12. Use Karnaugh maps to find a minimal expansion as a 
Boolean sum of Boolean products of Boolean functions 
that have as input the binary' code for each decimal digit 
rod produce as output a 1 if and only if the digit corre¬ 
sponding to the input is 

a) odd. 

b) not divisible by 3. 

c) not 4, 5, or 6. 

*13. Suppose that there are five members on a committee, 
but that Smith and Jones always vote the opposite of 
Marcus. Design a circuit that implements majority vot¬ 
ing of the committee using this relationship between 
votes. 

14. Use the Qtiine-McCIuskcy method to simplify the 
sum-of-produets expansions in Example 3. 

15. Use the Qtiine-McCIuskcy method to simplify the 
sum-of-produets expansions in Exercise 8. 

16. Use the Quine-McCluskey method to simplify the 
sum-of-products expansions in Example 4. 

17. Use the Qumc-McCl uskey method to simplify the 
sum-of-products expansions in Exercise 10. 

*18. Explain how Karnaugh maps can be used to simplify 
product-of-sums expansions in three variables. (Hint: 
Mark with a 0 all the maxterms in an expansion and 
combine blocks of maxterms, I 


19. Use the method from Exercise 18 to simplify the 
product-of-sums expansion (x + y + ;)(.v + y + z)<x + 
y + + > + z)(x -t y + 

*20, Draw a Karnaugh map for the 16 minterms in four 
Boolean variables on the surface of a torus. 

21. Build a circuit using OR gates, AND gates, and invert¬ 
ers that produces an output of 1 if a decimal digit, en¬ 
coded using a binary coded decimal expansion, is di¬ 
visible by 3, and an output of 0 otherwise. 

In Exercises 22-24 find a minimal sum-of-products expan¬ 
sion, given the Karnaugh map shown with don't care con¬ 
ditions indicated with <sfs. 



23. 
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Key Terms and Results 

TERMS 

Boolean variable: a variable that assumes only (lie values 
0 and 1 

x (complement of*): an expression with the value 1 when 
x has the value 0 and the value 0 when x has the value 1 
Jt v (at .t.v) (Boolean product or conjunction of x and 
y): an expression with ihe value 1 when both t and y 
have the value 1 and (he value 0 otherwise 
x 4 y (Boolean sum or disjunction of x and _v>: an expres¬ 
sion with the value 1 when either x or y. or both, has the 
value 1, and 0 otherwise 

Boolean expressions: the expressions obtained recur¬ 
sively by specifying that 0, f, xj,_ x„ are Boolean 

expressions and Fi, <F\ + £■> t, and (£]£;) are Boolean 
expressions if E\ and £; are 

dual of a Boolean expression: the expression obtained by 
interchanging + signs and ■ signs and interchanging Os 
and Is 

Boolean function of degree m a function from B n to S 
where B = {0, 1} 

Boolean algebra: a set B with two binary operations y 
and A, elements 0 and L and a complementation oper¬ 
ator that satisfies the identity, domination, associative, 
commutative, and distributive laws 
literal of the Boolean variable x: cither i or i 
minterm of x r , ., x„: a Boolean product y ( v- ■ ■ ■ >■„ 
where each y, is either t, or x t 
suni'of'producis expansion (or disjunctive normal 
form): the representation of a Boolean function as a 
disjunction of tninterms 

functionally complete: a set at Boolean operators is called 
functionally complete if every Boolean function can be 
represented using these operators 
x f y (or x NAND y): the expression that has Che value 0 
when both x and y have the value 1 and Ihe value 1 oth¬ 
erwise 


x i y (or x NOR y); the expression that has the value 0 
when either x ory or both have the value 1 and the value 
0 otherwise 

inverter: a device that accepts the value of a Boolean 
variable as input and produces the complement of the 
input 

OR gate: a device that accepts the values of two or more 
Boolean variables as input and produces their Boolean 
sum as output 

AND gate; a device that accepts the values of two or more 
Boolean variables as input and produces their Boolean 
product as output 

half adder: a circuit that adds two bits, producing a sum 
bit and a cany' bit 

full adder: a circuit that adds two bits and a carry, produc¬ 
ing a sum bit and a carry bit 

Karnaugh map for n variables: a rectangle divided into 
2" squares where each square represents a minterm in the 
variables 

RESULTS 

The identities for Boolean algebra (see Table 5 in Section 

y.-.o 

An identity between Boolean functions represented by 
Boolean expressions remains valid when the duals of 
both sides of the identity are taken. 

Every Boolean function can be represented by a xum-of- 
prodnets expansion. 

Each of the sets (-t, } arui {■, “} is functionally complete. 

Each of the sets {.1 } and {|} is functionally complete. 

The use of Karnaugh maps lo minimize Boolean expres¬ 
sions, 

The Qmne-McCIustey method for minimizing Boolean ex¬ 
pressions. 


Review Questions 


1. Define a Boolean function of degree n 

2. How many Boolean functions of degree 2 are there? 

3. Give a recursive definition of the set of Boolean expres¬ 
sions, 

4. a) What is the dual of a Boolean expression? 

b) What is the duality principle? How can it be used to 
find new identities involving Boolean expressions? 

5. Explain how to construct the sum-of-products expansion 
of a Boolean function. 


6, a) What does it mean for a set of operator* to be func¬ 

tionally complete? 

b) is the set {+, ■} functionally complete? 

c) Are there sets of a single operator that are function¬ 
ally complete? 

7, Explain how to build a circuit for a light controlled by 
two switches using OR gates, AND gates, and inverters. 

8, Construct a half adder using OR gates, A:\7J gates, and 
inverters. 
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9. Is there a single type of logic gate that can be used to 
build all circuits that can be built using OR gates, AND 
gates, and inverters? 

10. a) Explain how Karnaugh maps can be used to sim¬ 

plify sum-of-products expansions in three Boolean 
variables. 

b) Use a Karnaugh map to simplify the sum-of prod¬ 
ucts expansion xyz + xyz + xy z + aye + xyz. 

11. a) Explain how Karnaugh maps can be used to sim¬ 

plify sum-of-products expansions in four Boolean 
variables. 

b) Use a Karnaugh map to simplify the sum-of- 


products expansion wxyz+wxyz + ivxvz + wxy ; + 
H'.fvc + wx y • + + wxyz + wxvz. 

12, a) What is a don '! care condition ' 

b) Explain how dim) care conditions can be used to 
build a circuit using OR gates. AND gates, and in¬ 
verters that produces an output of 1 if a decimal 
digit is 6 or greater, and an output of 0 if this digit 
is less than 6. 

13. a) Explain how to use the Quine-MeCluskey method 

to simplify sum-of-produets expansions, 
b) Use this method to simplify xvz + xyz + xyz + 
xyz + xyz. 


Supplementary Exercises 


1. For which values of the Boolean variables x, y, and ; 
does 

a) x + v + ; = xyz’ b) x(y + z) = x + yzO 
c) x y z - X F v + ;? 

2. Let x and v belong to {0. 1}. Does it necessarily follow 

that x = y if there exists a value z in {0,1} such that 
a) a: ^ vz? b) x + z - y + z? 

e) JtQ; = y©z? d) iu- v J. 

e) x | ; = y ; z? 

A Boolean function F is called self-dual if and only if 

fi:.-([-rj = F(x[ .UI 

3. Which of the following functions are self-dual? 

a) F(x, y) = x It) Fix, y) = xy -+ xy 

c) Fix, y) = x + y d) Fix, y) = xy -*• xy 

4. Give an example of a self-dual Boolean function of 
three variables. 

*5. Ho w man y Boolean functions of degree n are self-dual ? 

We define the relation < on the set of Boolean functions 
of degree n so that F (1 where F and 0 are Boolean 
functions if and only if G(X[, xj,..., x„) = 1 whenever 
Ffxj.xi.xj -■ I 

6. Determine w hether F < G or G s t for the following 
pairs of functions. 

al F(x, v) = -V, G(x, y) = x + v 

b) Fi x, y ) - x + y, C(x, y) = xy 
0 F(x, y ) - x. G(x, v) = x + v 

7. Show that it F and G are Boolean functions of degree 
n. then 

a IfsFtf. b) FG s F. 

8. Show that if F, G. and H are Boolean functions of de¬ 
gree n, then F + G < W if and only if F s= H and 
G £ H. 

*9. Show that the relation *' is a partial ordering on the set 
of Boolean functions of degree n. 


*10. Draw the Hasse diagram for the poser, consisting of the 
set of the 16 Boolean functions of degree 2 (shown in 
fable 3 of Section 9.1) with the partial ordering - . 
*11. For each of the following equalities either prove it is 
an identity or find a set of values of the variables for 
which it does not hold. 

a) x [ (y \ z) = (x j y) 1 1 

b) x 1 (y l z) = (x l y) 1 (x l z) 

C) x l (y z) - (xi v) |(x|z) 

Define the Boolean operator G as follows: 101 = 1,10 
0 = 0,0 01 = 0,and0GO - 1. 

12. Show that x O y ~ xy + x y. 

13. Show that x 0 y = (x © yj. 

14. Show that each of the following identities holds, 

a) x O x = 1 b) x 0 5 = 0 

e) x G y = y O x 

15. Is it always true that (x Q y) O z = x O (y O z)? 

*16. Determine whether the set (0} is functionally com¬ 
plete. 

*17. How many of the 16 Boolean functions in two variables 
x and y can be represented using only the following set 
of operators, variables x and v, and values 0 and I'' 
a) { } b) {■} 

0 Uf d) {■. +} 

The notation for an XOR gate, which produces the output 
x © v from x and y, is as follows: 

7—t)D>*—* l ® >: 

18. Determine the output of circuits (a) and (b) shown at 
the top of the facing page. 

19. Show how a half adder can be constructed using fewer 
gates than are used in Figure I! of Section 9.3 when 
XOR gates can be used in addition to OR gates, AND 
gates, and inverters, 
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20, Design a circuit that determines whether three or more 
of four individuals on a committee vote yes on an issue, 
where each individual uses a switch for the voting. 

A threshold gate produces an output y that is either 0 
or I given a set of input values for the Boolean variables 

jti. ft,.A threshold gate has a threshold value T. 

which is a real number, and weights Wj, W;,..., ny, each of 
which is a real number. The output y of the threshold gate is 
I if and only if w, .ty * My ft !-■■■ + w n .r„ a T The thresh¬ 
old gate with threshold value T and weights Wi, tvi,.... w„ 
is represented by the following diagram. Threshold gates 
are useful in modeling in neurophysiology and in artificial 
intelligence. 



21. A threshold gate represents a Boolean function. Find 
a Boolean expression for the Boolean function repre¬ 
sented by the following threshold gate. 



22. A Boolean function that can be represented by a thresh¬ 
old gate is called a threshold function, Show that each 
of the following functions is a threshold function, 
a) Fix) = v b) Fix, y) = x + y 

c) Fix, y) = ftv d) F(x, y) = x | y 

e) F(x, y) = x ] y f) Fix, y, z) = x + yz 

g) F(w. X, >', z) = w + xy + z 

h) F[w, v, y. ;) = h'A'Z + fty; 

*23. Show that Fix, y) = x©y is not a threshold function. 
*24. Show that F(w, x, y, ;) = tet + yz is not a threshold 
function. 


Computer Projects 

WRIT!; PROGRAMS U I ill lift POLL OWING INPUT AND OUTFIT 


1. Given the values of two Boolean variables a andy. find 
the values of a + y. tv, r (I) r, a | v, and x , v. 

2. Construct a tabic listing the set of values of all 256 
Boolean functions of degree 3. 

3. Given the values of a Boolean function in n variables, 
where n is a positive integer, construct the sum-of- 
products expansion of this function. 


4. Given the table of values of a Boolean function, express 
this function using only the operators ■ and . 

5. Given the table of values of a Boolean function, express 
this function using only the operators + and - 

*6. Given the table of values of a Boolean function, express 
this function using only the operator |, 
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*7. Given the table of values of a Boolean function, express 
this function using only the operator j. 

8, Given the table of values of a Boolean function of 
degree 3, construct its Karnaugh map, 

9. Given the table of values of a Boolean function of 
degree 4, construct its Karnaugh map. 

** 10 . Given the table of values of a Boolean function, use 


the Quine-McCluskey method to find a minimal sum- 
of-products representation of this function. 

11 . Given a threshold value and a set. of weights for a 
threshold gate and the values of the n Boolean variables 
in the input, determine the output of this gate. 

12 . Given a positive integer n, construct a random Boolean 
expression in n variables in disjunctive normal form. 


Computations and Explorations 

l.rSH A COMPUTATIONAL PROGRAM OR PROGRAMS YOU HAVE WRITTEN TO DO THE FOLI .OWING EXERCISES. 


1. Compute the number of Boolean functions of degrees 7. 
8, 9, and 10. 

2. Construct a table of the Boolean functions of degree 3. 

3. Construct a table of the Boolean functions of degree 4. 

4. Express each of the different Boolean expressions in 
three variables in disjunctive normal form with just the 
NAND operator, using as few 1 VA 1 VD operators as possi¬ 
ble. What is the largest number of NAND operators re¬ 
quired? 

5. Express each of the different Boolean expressions in 
disjunctive normal form in four variables using 


just the NOR operator, with as few NOR operators as 
possible. What is the largest number of NOR operators 
required 9 

6. Randomly generate It) different Boolean expressions 
in four variables and determine the average number 
of steps required to minimize them using the Quine- 
MeCluskey method. 

7. Randomly generate 10 different Boolean expressions in 
five variables and determine the average number of steps 
required to minimize them using the Quine-McCluskey 
method- 


Writing Projects 

RESPOND TO THE FOLLOWING QUESTIONS WITH ESSAYS USING OUTSIDE SOURCES. 


1. Describe some of the early machines devised to solve 
problems in logic, such as the Stanhope Demonstrator, 
Jevons’s Logic Machine, and the Marquand Machine. 

2, Explain the difference between combinational circuits 
and sequential circuits. Then explain ho w flip-flops are 
used to build sequential circuits, 

3, Define a shift register and discuss how shift registers 
are used. Show how to build shift registers using flip- 
flops and logic gates. 

4 , Show how multipliers can be built using logic gates. 

5. Find out how logic gates are physically constructed. 
Discuss whether NAND and NOR gates are used in 
building circuits. 

6. Explain how dependency notation can be used to 
describe complicated switching circuits. 


7. Describe how multiplexers are used to build switching 
circuits, 

8. Explain the advantages of using threshold gates to con¬ 
struct sw itchin g circuits. 111 ustrate this by using thresh¬ 
old gates to construct half and hill adders. 

9. Describe the concept of hazard-free switching t ircuily 
and give some of the principles used in designing such 
circuits. 

10. Explain how r to use Karnaugh maps to minimize func¬ 
tions of five or six variables. 

11 . Describe what is meant by the functional decompo¬ 
sition of a Boolean function of n variables and dis¬ 
cuss procedures for decomposing Boolean functions 
into a composition of Boolean functions with fewer 
variables. 
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Modeling Computation 




computers can perform many tasks. Given a task, two questions arise. The first 


is: Can it be carried out using a computer? Once we know that this first question 


Vy has an affirmative answer, we can ask the second question: How can the task 
be carried out? Models of computation are used to help answer these questions. 

We will study three types of structures used in models of computation, namely, 
grammars, finite-state machines, and Turing machines. Grammars are used to generate 
the words of a language and to determine whether a word is in a language. Formal lan¬ 
guages, which are generated by grammars, provide models for both natural languages, 
such as English, and for programming languages, such as Pascal, Fortran, Prolog, and 
C. In particular, grammars are extremely important in the construction and theory' of 
compilers. The grammars that we will discuss were first used by the American linguist 
.Noam Chomsky in the 1950s. 

Various types of finite-state machines are used in modeling. All finite-state ma¬ 
chines have a set of states, including a starting state, an input alphabet, and a transition 
function that assigns a next state to every pair of a state and an input. The states of 
a finite-state machine give it limited memory capabilities. Some finite-state machines 
produce an output symbol for each transition; these machines can be used to model 
many kinds of machines, including vending machines, delay machines, binary adders, 
and language recognizers. We will also study finite-state machines that have no output 
but do have final states. Such machines are extensively used in language recognition. 
The strings that are recognized are those which take the starting state to a final state. 
The concepts of grammars and finite-state machines can be tied together. We will char¬ 
acterize those sets that are recognized by a finite-state machine and show' that these are 
precisely the sets which arc generated by a certain type of grammar. 

Finally, we will introduce the concept of a Turing machine. We will show how 
Turing machines can be used to recognize sets. We will also show how Turing ma¬ 
chines can be used to compute number-theoretic functions. Finally, we will discuss the 
Church-Turing thesis, which states that every' effective computation can be carried out 
using a Turing machine. 


10.1 


Languages and Grammars 


INTRODUCTION 

Words in the English language can be combined in various ways. The grammar of En¬ 
glish tells us whether a combination of words is a valid sentence. For instance, the frog 
writes neatly is a valid sentence, since it is formed from a noun phrase, the frog, made 
up of the article the and the noun frog, followed by a verb phrase, writes neatly made 
up of the verb writes and file adverb neatly. We do not care that this is a nonsensical 
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statement, because we are concerned only with the syntax, or form, of the sentence, and 
not its semantics, or meaning. We also note that the combination of words swims quickly 
mathematics is not a valid sentence because it does not follow the rules of English 
grammar. 

The syntax of a natural language, that is, a spoken language, such as English, 
French. German, or Spanish, is extremely complicated, In fact, it does not seem possi¬ 
ble to specify all the rules of syntax for a natural language. Research in the automatic 
translation of one language to another has led to the concept of a forma) language, 
which, unlike a natural language, is specified by a well-defined set of rules of syntax. 
Rules of syntax arc important not only in linguistics, the study of natural languages, but 
also in the study of programming languages, 

Wc will describe the sentences of a formal language using a grammar. The use 
of grammars helps when we consider the two classes of problems that arise most fre¬ 
quently in applications to programming languages: (1) How can we determine whether 
a combination of words is a valid sentence in a formal language? (2) How can we gen¬ 
erate the valid sentences of a formal language? 

Before giving a technical definition of a grammar, we will describe an example of 
a grammar that generates a subset of English. This subset of English is defined using a 
list of rules that describe how a valid sentence can be produced. We specify that 

1. a sentence is made up of a noun phrase followed by a verb phrase; 

2. a noun phrase is made up of an article followed by an adjective followed by 
a noun, or 

3. a noun phrase is made up of an article followed by a noun; 

4. a verb phrase is made up of a verb followed by an adverb, or 

5. a verb phrase is made up of a verb; 

6. an article is a, or 

7. an article is thr, 

S, an adjective is large, or 
9. an adjective is hungry; 

10. a noun is rabbit, or 

11. a noun is mathematician; 

12. a verb is cats, or 
IS. a verb is hops; 

14. an adverb is quickly, or 

15. an adverb is wildly. 

From these rules we can form valid sentences using a series of replacements until no 
more rules can be used. For instance, we can follow the sequence of replacements: 

sentence 

noun phrase verb phrase 
article adjective noun verb phrase 
article adjective noun verb adverb 
the adjective noun verb adverb 
the large noun verb adverb 
the large rabbit verb adverb 
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the large rabbit hops adverb 
(he large rabbit haps quickly 

to obtain a valid sentence. It is also easy to see that some other valid sentences are: a 
hungry mathematician eats wildly, a large mathematician hops, the rabbit eats quickly, 
and so on. Also, we can see that the quickly eats mathematician is not a valid sentence. 


PH RASE-STRUCTURE GRAMMARS 


Before we give a formal definition of a grammar, we introduce a little terminology. 

DEFINITION 1. A vocabulary (or alphabet) V is a finite, nonempty set of 
elements called symbols, A word (or sentence) over V is a string of finite length 
of elements of V. The empty’ string or null string, denoted by A, is the string con¬ 
taining no symbols. The set of all words over V is denoted by V*. A language 
over V is a subset of V'. 


Note that A, the empty string, is the string containing no symbols. It is different from 
0, the empty set. It follows that {A} is the set containing exactly one string, namely, the 
empty string. 

Languages can be specified in various ways. One way is to list all the words in 
the language. Another is to give some criteria that a word must satisfy to be in the lan¬ 
guage. In this section we describe another important way to specify a language, namely, 
through the use of a grammar, such as the set of rules we gave in the introduction to 
this section. A grammar provides a set of symbols of various types and a set of rules 
for producing words. More precisely, a grammar has a vocabulary V, which is a set of 
symbols used to derive members of the language. Some of the elements of the vocab¬ 
ulary' cannot be replaced by other symbols. These are called terminals, and the other 
members of the vocabulary, which can be replaced by other symbols, are called nonter¬ 
minals, The sets of terminals and nontemunals are usually denoted by T and ;V, respec¬ 
tively. In the example given in the introduction of the section, the set of terminals is {a, 
the, rabbit, mathematician, hops, eats, quickly, wildly}, and the set of nonterminals is 
{sentence, noun phrase, verb phrase, adjective, article, noun, verb, adverb}. There 
is a special member of the vocabulary called the start symbol, denoted by S, which 
is the dement of the vocabulary that we always begin with. In the example in the in¬ 
troduction, the start symbol is sentence. The rules that specify when we can replace a 
string from V\ the set of all strings of elements in the vocabulary, with another string 
are called the productions of the grammar. We denote by w<y —>■ w\ the production 
which specifies that wy can be replaced by W]. The productions in the grammar given 
in the introduction of this section were listed. The first production, written using this 
notation, is sentence —- noun phrase verb phrase. We summarize with the following 
definition. 

DEFINITION 2, A phrase-structure grammar G — (V,T, S, P ) consists of a 
vocabulary V, a subset T of V consisting of terminal dements, a start symbol S 
from V, and a set of productions P, The set V - T is denoted by N. Elements of N 
are called nonterminal symbols. Every production in P must contain at least one 
nonterminal on its left side. 
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EXAMPLE 1 


EXAMPLE 2 


EXAMPLE 3 


EXAMPLE 4 


Let G - {V, T, 5, P) where V = {a, b. A, B, S}, T = {a, b}, S is the start symbol, and 
P = IS — ABa, A —* BB, B —* ab, AB —* f b}. G is an example of a phrase-structure 
grammar. ■ 


We will be interested in the words that can be generated by the productions of a 
phrase-structure grammar. 

DEFINITION 3, Let G = (V, T, S, P) be a phrase-structure grammar. Let % = 
Izor (Aat is, the concatenation of l, za, and r) and wj *= h x r te-jfe&gs over V. 
If zq z\ is a production of G, we say that w>i is directly derivable from wp 

and we write i»oO'»i. If wq, wj. w m n a 0< are strings over V such that 

wj.wi ow*, then we aay tint is derivable firm h^, 

md we write wr, Bw*. Thesetfuence of steps used to obtain w H from wo is called 
% derivation. 


The string .4 aba is directly derivable from ABa in the grammar in Example 1 since 
B — ab is a production in the grammar. The string abababa is derivable from ABa 
since ABa o Aaba a> BBaba a Bababa r"> abababa, using the productions B —- ab, 
A -> BB, B —* ab, and B ah in succession. ■ 

DEFINITION 4. Let G = {V, T, S, P) be a phrase*-stwctnre grammar . Tk Um- 
gmge generated by G (or the language <?f G), denoted by L(G), is the set of all 
strings of terminals tfc-i are derivable from the sta ing stateB. In other wares, 

L(G) - {W € r | S & wj. 


In the following two examples we find the language generated by a phrase- 
structure grammar, 


Let G be the grammar with vocabulary V = {A', A, a, b}, set of terminals T = {a, b], 
starting symbol S, and productions P - {5 -*■ aA, S b, A aa}. What is L(G), 
the language of this grammar? 

Solution: From the start state S we can derive aA using the production S —*■ aA. We 
can also use the production 5 —■ B to derive b. From aA the production A —■ aa can 
be used to derive aaa. No additional words can be derived. Hence 1(G) - {b, aaa}. ■ 


Let G be the grammar with vocabulary V = {A. 0, 1}, set of terminals T = {0, 1}, start¬ 
ing symbol S, and productions P = [5 - i 15.5 — 0}. What is UG), the language of 
this grammar? 

Solution; From S we can derive 0 using S -> 0, or 115 using S — 115. From 115 
we can derive either 110 or 11115. From 11115 we can derive 11110 and 1111115. 
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EXAMPLE 5 


EXAMPLE 6 


At any stage of a derivation we can either add two Is at the end of the string or ter¬ 
minate the derivation by adding a 0 at the end of the string. We surmise that L(G) - 

{0.110. 11110,111JII0_}, the set of all strings that begin with an even number of 1 s 

and end with a 0. This can be proved using an inductive argument that shows that after 
n productions have been used, the only strings of terminals generated are those consist¬ 
ing of n - 1 or fev.er concatenations of 11 followed by 0. (This is left as an exercise for 
the reader.) ■ 


The problem of constructing a grammar that generates a given language often 
arises. The next three examples describe problems of this kind. 


Give a phrase-structure grammar that generates the set {0"l n n ~ 0, 1,2,. ..}. 

Solution; Two productions can be used to generate all strings consisting of a string 
of 0s followed by a string of the same number of Is, including the null string. The 
first builds up successively longer strings in the language by adding a 0 at the start of 
the string and a 1 at the end. The second production replaces S with the empty string. 
The solution is the grammar G = (V, T, S, P), where V = {0, 1, S}, T - {0, 1}, S is the 
starting symbol, and the productions are 

5 — US 1 
S — A. 

The verification that this grammar generates the correct set is left as an exercise for the 
reader. ■ 


The last example involved the set of strings made up of 0s followed by 1 s, where 
the number of 0s and Is are the same. The next example considers the set of strings 
consisting of 0s followed by Is, where the number of 0s and Is may differ. 


Find a phrase-structure grammar to generate the set {(Y"\ n \ m and n are notmegative 
integers}. 

Solution; We will give two grammars G, and Ch that generate this set. This will illus¬ 
trate that two grammars can generate the same language. 

The grammar G\ has alphabet l ' - {5,0, 1}, terminals T = {0, 1}, and produc¬ 
tions 5 — OS, S “* SI, and 5’ A. G i generates the correct set, since using the first 
production m times puts m 0s at the beginning of the string, and using the second 
production n times puts n Is af the end of the string. The details of this verification 
are left to the reader. 

The grammar Gi has alphabet V = {X A, O, 1}, terminals T = {0, If, and produc¬ 
tions S 0 S.S l A, S — 1 1, .4 —* 1 A, A —* 1,5 —>* A, The details that this gram¬ 
mar generates the correct set are left as an exercise for the reader. ■ 


Sometimes a set that is easy to describe can be generated only by a complicated 
grammar. The next example illustrates this. 
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EXAMPLE 7 


web 


EXAMPLE 8 


web 


One grammar that generates the set {0 a r2 B ; n = 0, 1, 2,3,...} is O' - (V. 7, S, Pi 
with V - {0, 1, 2,5, A, B }, T - {0, 1,2}, starting state 5, and productions 5 —* OS A B. 
S —■ A, BA — ,47?, (M -*• 01, 14 11, itf -» 12. 2 B 22. We leave it as an 

exercise for the reader to show that this statement is correct. The grammar given is the 
simplest type of grammar that generates this set, in a sense that will he made dear later 
in this section, The reader may wonder where this grammar came from, since it seems 
difficult to come up with this grammar from scratch. It may be comforting to know that 
this grammar can be systematically constructed using techniques from the theory of 
computation that tire beyond the scope of this book, ■ 


TYPES OF PHRASE-STRUCTURE GRAMMARS 


Phrase-structure grammars can be classified according to the types of productions that 
are allowed. We will describe the classification scheme introduced by Noam Chomsky. 
In Section 10.4 we will see that the different types of languages defined in this scheme 
correspond to the classes of languages that can be recognized using different models of 
computing machines, 

A type 0 grammar lias no restrictions on its productions, A type 1 grammar can 
have productions only of the form wq — wq, where the length of wq is greater than 
or equal to the length of wq, or of the form vtq — A. A type 2 grammar can have 
productions only of the form nq wq, where nq is a single symbol that is not a termi¬ 
nal symbol. A type 3 grammar can have productions only of the form wq -+ w 2 with 
wq = .4 and either wq = uB or wq - a , where A and B are nonterminal symbols and 
a is a terminal symbol, or with uq = S and wq = A. 

From these definitions we see that every type 3 grammar is a type 2 grammar, every 
type 2 grammar is a type I grammar, and every type I grammar is a type 0 grammar. 
Type 2 grammars are called context-free grammars since a nonterminal symbol that 
is the left side of a production can be replaced in a string whenever it occurs, no matter 
what else is in the string. A language generated by a type 2 grammar is called a context- 
free language. When there is a production of the form /wqr —*■ Iwir (but nor of the 
form wq —* wq), the grammar is called type 1 or context-sensitive since wq can be 
replaced by wq only when it is surrounded by the strings 1 and r. Type 3 grammars are 
also called regular grammars. A language generated by a regular grammar is called 
regular. Section 10,4 deals with the relationship between regular languages and finite- 
state machines. The Venn diagram in Figure 1 Shows the relationship among different 
types of grammars. 


From Example 6 we know that {0" f l" 1 m, n ■- 0, 1,2,...} is a regular language, since 
it can be generated by a regular grammar, namely, the grammar G 2 in Example 6. ■ 


Avrani Noam Chomsky (bom 1918), Noam Chomsky, horn in Philadelphia, is the son of a Hebrew 
scholar. He received his B.A., M A„ and Ph.D. in linguistics, all from the University of Pennsylvania. He 
was tin the staff of the University of Pennsylvania from 1950 until iy,5i. in 1955 he joinedthe faculty 
at beginning his MAT, career leaching engineers French and Gentian. Chomsky is currently the 

Ferrari P, Ward Professor ol foreign languages and linguistics at M.I.T. He is known fur his many funda¬ 
mental contributions to linguistics, including the study of gram man;. Chomsky is also widely known tbi Ins 
outspoken political activism. 
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EXAMPLE 9 


EXAMPLE 10 



It follows from Example 5 that {0"1" | n = 0,1,2,...} is a context-free language, since 
the productions in this grammar are S -* OS 1 and S A, However, it is not a regular 
language. This will be shown in Section 10.4, ■ 


The set {0" l B 2 n [ n = 0, t, 2,...} is a context-sensitive language, sinee it can be gener¬ 
ated by a type 1 language, as Example 7 shows, but not by any type 2 language. (This 
is shown in Exercise 28 in the supplementary exercises at the end of the chapter.) ■ 


Table I summarizes the terminology used to classify phrase-structure grammars, 

DERIVATION TREES 

A derivation in the language generated by a context-free grammar can be represented 
graphically using an ordered rooted tree, called a derivation, or parse, tree. The root 
of this tree represents the starting symbol. The internal vertices of the tree represent the 
nonterminal symbols that arise in the derivation. The leaves of the tree represent the 
terminal symbols that arise. If the production A —»■ w arises in the derivation, where 
w is a word, the vertex that represents A has as children vertices that represent each 
symbol in w, in order from left to right. 


TABLE 1 Types of Grammars. 

Type 

Restrictions on Productions w/ — w 2 

0 

No restrictions 

1 

Kw \) S dk' 2 !. or hi = A 

2 

k'i = .4 where A is nonterminal symbol 

3 

H’| = A and ti j = aB or w 7 = a, where A F M, B £ .V, arid a € T, or S —> A 
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EXAMPLE U 


EXAMPLE 12 


noun phrase 


verb phrase 


article adjective noun verb adverb 


the hungry rabbit eats quickly 


FIGURE 2 A Derivation Tree. 


Construct a derivation tree for the derivation of the hungry rabbit eats quickly, given in 
the introduction of this section. 

Solution: The derivation tree is shown in Figure 2. ■ 


The problem of determining whether a string is in the language generated by a 
context-free grammar arises in many applications, such as in the construction of com¬ 
pilers. Two approaches to this problem are indicated in the follow ing example. 

Determine whether the word chub belongs to the language generated by the grammar 
C = (K 7. S, P) where V = {a, b, c. A, B, C, S}, T - {a, b, c}, S is the starting symbol, 
and the productions are 

S AB 
A - Ca 
B-* Ba 
B -» Cb 
B -* b 
C. -» cb 
C~*b. 

Solution: One way to approach this problem is to begin with S and attempt to derive 
cbab using a series of productions. Since there is only one production with S on its left- 
hand side, we must start with S o AB. Next we use the only production that has A on its 
left-hand side, namely A —» Ca, to obtain S © AB © CaB, Since cbab begins with the 
symbols cb, we use the production C —* cb. This gives us S o Ah o CaB © cbaB. 
We finish by using the production B —> b, to obtain S & AB n CaB o cbaB © cbab. 
The approach that we have used is called top-down parsing, since it begins with the 
starting symbol and proceeds by successively applying productions. 

There is another approach to this problem, called bottom-up parsing. In this ap¬ 
proach, we work backward. Since cbab is the string to be derived, we can use the pro¬ 
duction C —* cb. so that Cab o cbab. Then, we can use the production A —» Ca, so 
that Ab © Cab o cbab. Using the production B — b, gives AB © Ah © Cab © 
cbab. Finally, using S —* AB shows that a complete derivation for cbab is S o AB 
Or Ab a> Cab o cbab, g 
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BACKl S-SAIIR FORM 

There is another notation that is sometimes used to specify a type 2 grammar, called the 
Baekus-Naur form, after John Backus, who invented it, and Peter Naur, who refined 
it for use in the specification of the programming language ALGOL. The Baekus-Naur 



form is used to specify the syntactic rules of many computer languages, including Java. 
The productions in a type 2 grammar have a single nonterminal symbol as their left’ 
hand side. Instead of listing all the productions separately, we can combine all those 
with the same nonterminal symbol on the left-hand side into one statement. Instead of 
using the sy mbol —* in a production, we use the symbol:: =. We enclose all nonterminal 
symbols in brackets, {). and we list all the right-hand sides of productions in the same 
statement, separating them by bars. For instance, the productions A — Aa, A —* a, 
and A -» AB can be combined into (A):: = {A}a J a j {A)(B}, 

EXAMPLE 13 

What is the Baekus-Naur form of the grammar for a subset of English described in the 
introduction to this section? 

Solution: The Baekus-Naur form of this grammar is: 

{ sentence ) ;:= {noun phmse){verbphrase) 

{noun phrase) (artide}(adjecitve){nourt) \ (arttcle){noun) 

{verb phrase)(verb){adverb) j (verb) 

{article) :: = a the 
{adjective) :: = large hungry 
{noun) :: = rabbit \ mathematician 
{verb):: - eats \ hops 

(adverb)::— quickly \ wildly 1 

EXAMPLE 14 

Give the Baekus-Naur form for the production of signed integers in decimal notation. 
(A signed integer is a nonnegative integer preceded by a plus sign or a minus sign.) 

web 

John Backus (born 1924). John Backus was bom in Philadelphia. He received his bachelor of science and 
master’s degree in mathematics front Columbia University. Backus joined IBM as a programmer in 1950, 
He participated in the design and development of two of IBM's early computers. From 1954 to 1958 he led 
the IBM group that developed FORTRAN. Backus became a staff member at the IBM Watson Research 
Center in 1958. He was part of the committees that designed the programming language ALGOL, using 
what is now called the Baekus-Naur form for the description of the syntax of this language. Later, Backus 

web 

worked on the mathematics of families of sets and on a functional sty le of programming. Backus became 
an IBM Fellow in 1965, and he received the National Medal of Science in 1974 and the prestigious Turing 
Award from the Association of Computing Machinery in 1977. 

Pder Naur (born 1928). Peter Naur was horn in Frederiksberg, near Copenhagen. As a boy be became 
interested in astronomy. Not only did he observe heavenly bodies, but he also computed the orbits of comets 
and asteroids. Naur attended Copenhagen University, receiving his degree in 1949. He spent 1950 and 
1951 in Cambridge, where he used an early computer to calculate the motions of comets and planets. After 
returning to Denmark he continued working in astronomy but kept his ties to computing. In 1955 he served 
as a consultant to the building of the first Danish computer. In 1959 Naur made the switch from astronomy 
to computing as a full-time activity. His first job as a full-time computer scientist was participating in the 
development of the programming language .ALGOL. From I960 to 1967 he worked on the development 
of compilers for ALGOL and COBOL, In 1969 he became professor of computer science at Copenhagen 
University, where he has worked in the area of programming methodology. 
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Solution: The Baekus-Naur form for a grammar that produces signed integers follows: 

(signed integer) (sign){inieger) 

(sign) :;= + j - 

(integer) :;= {digit} | (digit)(integer) 

(digit) :— 01 I | 2 j 3 j 4 | 5 | 6 | 7 j 8 1 9 a 


Exercises 

Exercises 1-3 refer to the grammar with Start symbol sen¬ 
tence, set of terminals T = {the, sleepy, happy, tortoise, 
hare, passes, runs, quickly, slowly}, set of nonterminals 

N = [noun phrase, transitive verb phrase, intransi¬ 
tive verb phrase, article, adjective, noun, verb, adverb}, 

and productions. 

sentence —■ noun phrase transitive verb phrase 
noun phrase 

sentence noun phrase intransitive verb phrase 

noun phrase —*• article adjective noun 

noun phrase ^ article noun 

transitive verb phrase transitive verb 

intransitive verb phrase intransitive verb adverb 

intransitive verb phrase -*■ intransitive verb 

article —* the 

adjective ■ -* sleepy 

adjective happy 

noun —*■ tortoise 

noun — hare 

transitive verb passes 

intransitive verb runs 

adverb * quickly 

adverb —*■ slowly 

1. Use the set of productions to show that each of the 
following is a valid sentence. 

a) the happy hare runs 

b) the sleepy tortoise runs quickly 
e) the tortoise passes the hare 

d| the sleepy fuire passes the happy tortoise 

2. Find live other valid sentences, besides those given in 
Exercise 1. 

3. Show 1 that the hare runs the sleepy tortoise is not a valid 
sentence. 

*4. Let V - {S.A.B.a.b } and T = {a,b}. Find the 
language generated by the grammar {K T, S, f’j when 
the set P of products consists of 

a) S AB. A — ah, B —> bb. 

b) $ — AB, S aA, A — 1 • a, B —* ba. 

c) S — AB. S — A A, A — aB, A — ah, B -> b. 

d) S — 1 A A. S B, A —* wA , A —* aa, 

B — hB. B -- b. 


e) S — Afi, A - a Ah, B -<■ hBa, A — A, B — A 

5. Construct a derivation of 0 3 1 3 using the grammar given 
in Example 5. 

6. Show that the grammar given in Example 5 generates 
thesetfO")' 1 | n = 0. b 3,...}. 

7. a) Construct a derivation of 0 2 1 4 using the grammar 

0 1 in Example 6 

b) Construct a derivation of (H 4 using the grammar 
dr in Example 6. 

8. a) Show that the grammar Gj given in Example 6 

generates the set {0™ 1” | m, n = 0, 1,2,...}. 
b) Show that the grammar G? in Example 6 generates 
the same set. 

9. Construct a derivation of 0 ; 1 2 2 : in the grammar given 
in Example 7. 

*10. Show that the grammar given in Example 7 generates 
the set fO" I "2* jn = 0, 1,2,...}. 

*11. Find a phrase-structure grammar for each of the follow¬ 
ing languages. 

a) the set of all bit strings containing an even number 
nflk and no U 

b) the set of all bit strings made up of a 1 followed by 
an odd number of 0s 

c) the set of all bit strings containing an even number 
of 0s and an even number of 1 s 

d) the set of all strings containing 10 or more (Is and 
no Is 

e) the set of all strings containing more 0s than Is 

f) the set of all strings containing an equal number of 
0sand Is 

gj the set of all strings containing an unequal number 
of 0s and 1 s 

12, Construct phrase-structure grammars to generate each 
of the following sets, 

a) {01 s " , n s 0} 
b} {0"l s " | n s 0} 
c) {0*1 "'0' 1 I rn 2= (I, n > 0} 

13. Lei V = {S.A,B,a,h\ and T = {& b\. Determine 
whether G - (V, T, S, Pi is a type 0 grammar but not 
a type 1 grammar, a type 1 grammar but not a type 2 
grammar, or a type 2 grammar but not a type 3 grammar 
if P, the set of productions, is 
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a) S d,46, A Bb. B A. 

b) S ■ ■* (7,4, A — a, A — h. 

c) S — .45a, 46 — a. 

d ) S — 454, A -* nB, B — ab. 


e) 

s - 

- hA, A - 

- 6, B - 

-* a. 

f) 

S —» a A, a A 

-* B, B 

a A, A —*i 

K) 

s - 

1 

5 th 

tn 

1 

* A. 

h) 

s - 

- 46, B - 

- a Ah, aAb ~ 1 b. 

i) 

S - 

aA, A —■ hB , B 

— h, B — A. 

j> 

S — 4, A -* 

B , B 

A. 


14. A palindrome is a string that reads the same backward 
as it does forward, that is, a string w where w = vv*, 
where n ,if is the reversal of the string m . Find a context- 
free grammar that generates the set of ail palindromes 
over the alphabet {0. 1}. 

*15, I -d 6' ; and C: be context-free grammars, generating 
the languages L(G l ) and l(G z ). respectively. Show that 
there is a context-free grammar generating each of the 
following sets. 

a) L{G|) LJ Utip b) L(G\)UG Z ) 
e) 6(Gi)" 

16. Find the strings constructed using the derivation trees 
shown below. 


sentence 



noun phrase verfr phrase 

XIX /\ 

article adjective noun verb adverb 


a large mathematician hops wildly 


signed integer 



sign integer 



digit integer 



11 digit integer 


8 digit 


17. Construct derivation trees for the sentences in Ex¬ 
ercise 1. 

18, Let G be the grammar with V' = \a, b, e, 5}, T = 
jo, b, r}, starting symbol. S, and productions S —* abS, 
S bcS , S —• bhS, S -* a, S -» cb- Construct 
derivation trees for 

a) hchha, 

b) bbbebba, 

c) bcahbbbhcb. 

* 19. Use top-down parsing to determine whether each of the 
following strings belongs to the language generated by 
the grammar in Example J 2. 

a) baba b) abab 

c) i-baha d) bhbeha 

*26. Use bottom-up parsing to determine whether the 
strings in Exercise 3 9 belong to the language generated 
by the grammar in Example 12, 

21. Construct a derivation tree for -109 using the grammar 
givenin Example 14. 

22. a) Explain what the productions are in a grammar if 

the Backus-Naur form for productions is as fol¬ 
lows? 

(expression) ::= {(expression)) \ 

{expression) + (expression) j 
(expression) * (expression) 
(variable} 

(variable ) = x j y 

h) Find a derivation tree for (at * y) + x in this gram¬ 
mar. 

23. a) Construct a phrase-structure grammar that gen¬ 

erates all signed decimal numbers, consisting of 
a sign, either + or a nonnegative integer; 
and a decimal fraction that is either the empty 
string or a decimal point followed by a posi¬ 
tive integer, where initial zeros in an integer are 
allowed. 

b) Give the Backus-Naur form of this grammar. 

c) Construct a derivation tree for -31,4 in this 
grammar. 

24. a) Construct a phrase-structure grammar for the sei of 

a!) fractions of the form alb, where a is a signed 
integer in decimal notation and b is a positive 
integer. 

b) What is the Backus-Naur form for this grammar? 

c) Construct a derivation tree for +311/17 in this 
grammar. 

25. Let G be a grammar and let R be the relation containing 
the ordered pair (u'o, wj J if and only if wv is directly 
derivable from tvu in G. What is the reflexive transitive 
closure of 5? 
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10.2 _ 

Finite-State Machines with Output 

INTRODUCTION 


Many kinds of machines, including components in computers, can be modeled using a 
structure called a finite-state machine, Several types of finite-state machines are com¬ 
monly used in models. All these versions of finite-state machines include a finite set 
of states, with a designated starting state, an input alphabet, and a transition function 
that assigns a next state to every state and input pair. In this section we will study 
those finite-state machines that produce output. We will show how finite-state machines 
can be used to model a vending machine, a machine that delays input, a machine that 
adds integers, and a machine that determines whether a bit string contains a specified 
pattern. 

Before giving formal definitions, we will show how a vending machine can be mod¬ 
eled, A vending machine accepts nickels (5 cents), dimes (10 cents), and quarters (25 
cents), When a total of 30 cents or more has been deposited, the machine immediately 
returns the amount in excess of 30 cents. When 30 cents has been deposited and any 
excess refunded, the customer can push an orange button and receive an orange juice or 
push a red button and receive rut apple juice, V/e can describe how the machine works 
by specifying its states, how it changes states when input is received, and the output 
that is produced for every' combination of input and current state. 

The machine can be in any of seven different states a\, i = 0,1, 2,_6, where s; is 

the state where the machine has collected 5i cents. The machine starts in state s 0t with 
0 cents received. The possible inputs are 5 cents, 10 cents, 25 cents, the orange button 
(Q), and the red button (/?). The possible outputs are nothing (n), 5 cents, 10 cents, 15 
cents, 20 cents, 25 cents, an orange juice, and an apple juice. 

We illustrate how this model of the machine works with the following example. 
Suppose that a student puts in a dime followed by a quarter, receives 5 cents back, 
and then pushes the orange button for an orange juice. The machine starts in state %, 
The first input is 10 cents, which changes the state of the machine to and gives no 
output. The second input is 25 cents. This changes the state from s~ A to arid gives 5 
cents as output. The next input is [he orange button, which changes the state from ,sg 
back to s 0 (since the machine returns to the start state) and gives an orange juice as its 
output. 

We can display all the state changes and output of this machine in a table. To do 
this we need to specify for each combination of state and input the next state and the 
output obtained. Table 1 shows the transitions and outputs for each pair of a state and 
an input. 

Another way to show the actions of a machine is to use a directed graph with labeled 
edges, where each state is represented by a circle, edges represent the transitions, and 
edges are labeled with the input and the output for that transition. Figure 1 shows such 
a directed graph for the vending machine. 


FINITE-STATE MACHINES WITH OUTPUTS 


We will now give the formal definition of a finite-state machine with output. 
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TABLE 1 State Table for a Vending Machine, 

State 

Next State 

Output 

5 

HI 

Input 

25 

0 

R 

5 

Hi 

Input 

25 0 

A 

so 

St 

$2 

ft 

ft 

ft 

n 

n 

n 

n 

n 

■i'J 

.Ft 

ft 

Sb 

ft 

Si 

n 

n 

n 

n 

n 


*3 

Si 

i'6 


s: 

n 

n 

5 

n 

n 

ft 

ft 

Si 

ft 

Si 

- ? 3 

n 

n 

10 

n 

n 

J* 

St. 

ft 

ft 

ft 

ft 

n 

n 

15 

n 

n 

■'5 

ft 

■ft 

Sb 

ft 

ft 

n 

5 

20 

n 

n 

ft 

Si 

*6 

ft 

ft 

^0 

5 

10 

25 

OJ 

AJ 


DEFINITION I. A finite-state machine M = (S, /, O, f, g, so) consists of a finite 
set S of states, a finite input alphabet I, a finite output alphabet 0 , a transition junc¬ 
tion fth&l assigns to each state and input pair a new state, an output function g Asa 
assigns to each state and input pair an output, and an initial state % 


Let M = (S ,!, 0, f, g, i'o) be a finite-state machine. We can use a state table to 
represent the values of the transition function/' and the output function g for all pairs 
of states and input. We previously constructed a state table for the vending machine 
discussed in the introduction to this section. 


EXAMPLE 1 The state table shown in Table 2 describes a finite-state machine with 5 = {so, sj, j?. -> 3 }, 

1 — {0, 1}, and O = [0. 1}. The values of the transition function / are displayed in the 
first two columns, and the values of the output function g are displayed in the last two 
columns, ■ 


y.Ji 35.5 25.20 
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EXAMPLE 2 


EXAMPLE 3 



FIGURE 2 The State Diagram for the 
Finite-State Machine Shown in Tabic 2. 


TABLE 2 


/ 



Input 

Input 

State 

0 1 

0 1 

*0 

*1 s 0 

1 0 


S3 S(l 

1 1 

n 

Si Si 

0 1 

■'3 

S2 S| 

0 0 


Another way to represent a finite-state machine is to use a state diagram, which 
is a directed graph with labeled edges. Tn this diagram, each state is represented by a 
circle. Arrows labeled with the input and output pair are shown for each transition. 


Construct the state diagram for the finite-state machine with the state table shown in 
Table 2, 

Solution: The state diagram for this machine is shown in Figure 2. ■ 

Construct the state table for the finite-state machine with the state diagram shown in 
Figure 3. 

Solution: The state table for this machine is shown in Table 3, ■ 


An input string takes the starting state through a sequence of states, as determined 
by the transition function. As we read the input string symbol by symbol (from left to 
right), each input symbol takes the machine from one state to another. Because each 
transition produces an output, an input string also produces an output string. 


TABLE 3 


/ 

g 


Input 

Input 

State 

0 1 

0 1 

so 

S| S] 

I 0 

Si 

Si .Sj 

1 1 

S'2 

■Sj S4 

0 0 

■S3 

S| So 

0 0 

S4 

S 3 J'4 

0 0 
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EXAMPLE 4 


EXAMPLE 5 


Suppose that the input string is x — xjx? --x*. Then, reading this input takes 
the machine from state sg to state .?], where = f[sg, X\ ), then to state sj, where 
S 2 - f(s\, x 2 ), and so on. ending at states* = /(j*_ 1 , x*}, This sequence of transitions 
produces an output string v ! i > r 2 ' ■ ■y*, where Vi - gis'u, xj) is the output corresponding 
to the transition from Sr, to x !t y% = gisu x 2 ) is the output corresponding to the transition 
from si to s 2 > told so on. In general yy - g(sj-\, x,) for j = 1,2,.... k. Hence, we 
can extend the definition of the output function g to input strings so that g(x) - v, 
where y is the output corresponding to the input string x. This notation is useful in 
many applications. 


Find the output string generated by the finite-state machine in Figure 3 if the input 
string is 101011. 

Solution: The output obtained is 001 (MX). The successive states and outputs are shown 
in Table 4. ■ 


We can now give some examples of useful finite-state machines. These examples 
illustrate that the states of a finite-state machine give it limited memory capabilities. 
The states can be used to remember the properties of the symbols that have been read 
by the machine. However, since there are only finitely many different states, finite- 
state machines cannot be used for some important purposes. This will be illustrated in 
Section 10.4. 


An important element in many electronic devices is a unit-delay machine, which pro¬ 
duces as output the input string delayed by a specified amount of time. How can a 
finite-state machine be constructed that delays an input string by one unit of time, that 
is, produces as output the bit string Oxi x 2 • • * .x> -1 given the input bit string Xi x 2 ■ • ■ x*? 

Solution: A delay machine can be constructed that has two possible inputs, namely, 
0 and l. The machine must have a start state i- 0 . Since the machine has to remember 
whether the previous input was a 0 or a 1, two other states s j and are needed, where 
the machine is in state S] if the previous input was 1 and in state jj if the previous input 
was 0. An output of 0 is produced for the initial transition from % Each transition from 
j'i gives an output of 1, and each transition from r 2 gives an output of 0. The output 
corresponding to the input of a string Xi ■ ■ • X* is the string that begins with 0, followed 
by Xi, followed by x% .. ending with x*_i. The state diagram for this machine is 
show n in Figure 4. ■ 


TABLE 4 

Input 

1 

0 

1 

0 

l 

i 

- 

State 

«D 

Sj 

i'| 

S2 

■S} 

Ai 

■S3 

Output 

0 

(J 

1 

0 

0 

0 

- 
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EXAMPLE 6 


EXAMPLE 7 


Produce a finite-state machine that adds two integers using their binary expansions. 

Solution: When u„ ■ ■ - .v ( .tc )2 and (y„ • • ■ >' 1 yo )2 are added, the following procedure 
(as described in Section 2.4) is followed. First, the bits xg and yo are added, producing 
a sum bit 20 and a carry bit . This carry bit is either 0 or 1. Then, the bits x\ and y\ 
are added, together with the cany c 0 - This gives a sum bit Z\ and a cany bit c t . This 
procedure is continued until the nth stage, where x n , y n , and the previous carry e„-i 
are added to produce the sum bit z, n and the carry bit c„, which is equal to the sum bit 

A finite-state machine to carry out this addition can be constructed using just two 
states. For simplicity we assume that both the initial bits x a and y„ are 0 (otherwise 
we have to make special arrangements concerning the sum bit The start state r 0 
is used to remember that the previous cany' is 0 (or for the addition of the rightmost 
bits), The other state, S|, is used to remember that the previous carry is 1. Since the 
inputs to the machine are pairs of bits, there are four possible inputs. We represent these 
possibilities by 00 (when both bits are 0), 01 (when the first bit is 0 and the second is 
I), 10 (when the first bit is 1 and the second is 0), and 11 (when both bits are 1), The 
transitions and the outputs are constructed from the sum of the two bits represented by 
the input and the carry represented by the state. For instance, when the machine is in 
state j, and receives 01 as input, the next state is s, and the output is 0, since the sum 
that arises is 0 + 1 + 1 = (1 0 ) 2 . The state diagram for this machine is shown in Figure 5. 


In a certain coding scheme, when three consecutive Is appear in a message, the receiv¬ 
er of the message knows that there has been a transmission error. Construct a finite- 
state machine that gives a 1 as its output bit if and only if the last three bits received 
are all Is. 

Solution: Three state s are needed in this machine. The start state xq remembers that the 
previous input value, if it exists, was not a 1. The state j| remembers that the previous 
input was a 1, but the input before the previous input, if it exists, was not a 1. The state 
remembers that the previous two inputs were is. An input of i takes to j|, since 
now a 1, and not two consecutive Is, has been read; it takes j] to si, since now two 
consecutive Is have been read; and it takes s 2 to itself, since at least two consecutive Is 
have been read. An input of 0 takes every' state to r 0 , since this breaks up any string of 
consecutive 1 s. The output for the transition from s 2 to itself when a 1 is read is I, since 
this combination of input and state show that three consecutive Is have been read. All 
other outputs are 0. The state diagram of this machine is shown in Figure 6, ■ 
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FIGURE 6 A Finite-State Machine That Gives an Output of 1 If and Only If the Input 
String Read So Far Ends with 111. 


The machine in Figure 6 is an example of a language recognizer, because it pro¬ 
duces an output of 1 if and only if the input string read so far has a specified property. 
Language recognition is an important application of finite-state machines. 

Types of Finite-State Machines Many different kinds of finite-state machines have 
been developed to model computing machines. In this section we have given a definition 
of one type of finite-state machine. In the type of machine introduced in this section, 
outputs correspond to transitions between states. Machines of this type are known as 
Mealy machines, since they were first studied by G. H. Mealy in 1955. There is another 
important type of finite-state machine with output, where the output is determined only 
by the state. This type of finite-state machine is known as a Moore machine, since E. 
F. Moore introduced this type of machine in 1956. Moore machines are considered in a 
sequence of exercises at the end of this section. 

In Example 7 we showed how a Mealy machine can be used for language recogni¬ 
tion. However, another type of finite-state machine, giving no output, is usually used for 
this purpose. Finite-state machines with no output, also known as finite-state automata, 
have a set of final states and recognize a string if and only if it takes the start state to a 
final state. We will study this type of finite-state machine in Section 10.3. 


Exercises 


1. Draw the state diagrams for the finite-state machines 
with the following state tables. 


a) 



/ 

IT 


Input 

Input 

State 

0 1 

0 I 

so 

' S] *0 

0 1 

*1 

.'0 S’ 

0 1 

S2 

1 St *1 

0 0 


Suite 

/ 

i 

Input 
0 1 

Input 

0 I 

so 

S| So 

0 0 

Si 

S'2 Jll 

1 1 

52 

So S ) 

0 1 

SI 

S| S 2 

1 0 


State 

f 

8 

Input 

0 I 

Input 

0 I 

so 

So Sj 

1 1 


sn s? 

0 1 

Sl 

So S; 

0 o 


S[ S] 

l 1 

S'4 

Si So 

1 0 
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2. Give the state tables for the finite-state machines with 
the following state diagrams. 



3, Given the finite-state machine shown in Example 2, 
determine the output for each of the following input 
strings. 

al 0111 h) 11011011 e) 61010101010 

4, Given the finite-state machine shown in Example 3, 
determine ihe output for each of the following input 
strings. 

a) 0000 b) I01010 c) 11011100010 

5. Construct a finite-state machine that models a soda ma¬ 
chine which accepts nickels, dimes, and quarters. The 
soda machine accepts change until 35 cents has been 
put in. It gives change back for any amount greater than 
35 cents. Then the customer can push buttons to receive 
either a cola, a root beer, or a ginger ale. 

6. Construct a finite-state machine that models a news¬ 
paper vending machine which has a door that can be 
opened only after either three dimes (and any number 
of other coins) or a quarter and a nickel (and any num¬ 


ber of other coins) have been inserted. Once the door 
can be opened, the customer opens it and takes a paper, 
dosing the door. No change is ever returned no mat¬ 
ter how much extra money has been inserted The next 
customer starts with no credit. 

7. Construct a finite-state machine that delays an input 
siring two hits, giving (X) as the first two bits of output. 

8. Construct a finite-state machine that changes every 
other bit, starting with the second bit, of an input string, 
and leaves the other bits unchanged. 

9. Construct a finite-state machine for the log-1 on proce¬ 
dure fpr a computer, where the user logs in by entering a 
user identification number, which is considered to he a 
single input, and then a password, which is considered 
to be a single input. If the password is incorrect, the 
user is asked for the user identification number again. 

10. Construct a finite-state machine for a combination lock 
that contains numbers 1 through 40 and that opens only 
when the correct combination, K) right, 8 second left, 
37 right, is entered. Each input is a triple consisting of 
a number, the direction of the turn, and the number of 
times the lock is turned in that direction, 

11. Construct a finite-state machine for a toll machine that 
opens a gate after 25 cents, in nickels, dimes, or quar¬ 
ters, has been deposited. No change is given for over¬ 
payment, and no credit is given to the next driver when 
more than 25 cents has been deposited. 

12. Construct a finite-state machine that gives an output of 
1 if the number of input symbols read so far is divisible 
by 3 and an output of 0 otherwise. 

13. Construct a finite-state machine thaL determines 

whether the input string has a 1 in the last position 

and a 0 in the third to the last position read so far. 

14. Construct a finite-state machine that determines 

w hether the input string read so far ends in at least 
five consecutive Is. 

15. Construct a finite-stale machine that determines 

whether the word computer has been read as the last 
eight characters in the input read so far. where the input 
can be any string of English letters, 

A Moore machine M = (S, /, O. /. g, r D ) consists of a fi¬ 
nite set of states, an input alphabet /, an output alphabet 
O, a transition function / that assigns a next state to ev¬ 
ery pair of a state and an input, an output function g that 
assigns an output to every state, and a starting state ,r ft . A 
Moore machine can he represented either by a table listing 
the transitions for each pair of state and input and the out¬ 
puts for each state, or by a state diagram that displays the 
states, the transitions between states, and the output foreach 
state. In the diagram, transitions are indicated with arrows 
labeled with the input, and the outputs are shown next to 
the states. 

16. Construct the state diagram for the Moore machine 
with the following state table. 
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/ 


5 tote 

input 

0 1 

8 


i'O s l 

0 

->’l 

St, .To 

1 


Si Jf[ 

1 

■ri 

■'2 -bi 

1 


17, Construct the state tabic tor the Moore machine 

with the state diagram shown at the top of" the next 
column. Each input string to a Moore machine Af 
produces an output string. In particular, the output 
corresponding to the input string • u t is the 

string where a, = f{s M.a,) for 

t = 1,2. k. 

18, Find the output string generated by the Moore machine 

in Exercise 16 with each of the following input strings, 
a) 0101 b) 111111 cMIIOIllOin 

19, Find the output string generated by the Moore machine 


0 



in Exercise 37 with each of the input strings in Exer¬ 
cise 18. 

20. Construct a Moore machine that gives an output of 1 
whenever the number of symbols in the input string 
read so far is divisible by 4. 

21. Construct a Moore machine that determines whether an 
input string contains an even or odd number of Is, The 
machine should give 1 as output if an even number of 
Is arc in the string and 0 as output if an odd number 
of Is are in the string. 


103 _ 

Finite-State Machines with No Output 


INTRODUCTION 


One of the most important applications of finite-state machines is in language recogni- 
tion. This application plays a fundamental role in the design and construction of compil¬ 
ers for programming languages. In Section 10.2 we showed that a finite-state machine 
with output can he. used to recognize a language, by giving an output of 1 when a string 
from the language has been read and a 0 otherwise. However, there are other types of 
finite-state machines that are specially designed for recognizing languages. Instead of 
producing output, these machines have final states. A string is recognized if and only 
if it takes the starting state to one of these final states. 


SET OF STRINGS 


Before discussing finite-state machines with no output, we will introduce some impor¬ 
tant background material on sets of strings. The operations that will be defined here w ill 
be used extensively in our discussion of language recognition by finite-state machines. 


DEFINITION 1. Suppose that A and B are subsets of V\ where Vis a vocab ular y 
The coocatenatHMR of A and B, denoted by AB, is the set of all st rings of the form 
xy where x is a string in A and y is a string in B. 
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EXAMPLE 1 Let 4 = {0, 11} and B - {1, 10, 110}. Find AB and BA. 

Solution: The set AB contains ever)' concatenation of a string in A and a string in B. 
Hence, Ail = {01,010,0110, 111,1110,11110}. The set 5/1 contains every concatena¬ 
tion of a string in B and a string in 4. Hence, BA = {10, 111, 100,1011,1100, 11011}, 


Note that it is not necessarily the case that AB = BA when A and B are subsets of V\ 
where V is an alphabet, as Example 1 illustrates. 

From the definition of the concatenation of two sets of strings, we can define A' ! , 
for n - 0,1,2,.... This is done recursively by specifying that 

4° = {4 

A" +! = A*A for?! = 0,1,2,.... 

EXAMPLE 2 Let A = {1,00}. Find A" for n = 0,1,2, and 3. 

Solution: We have A 0 = {A} and A 1 = A 11 A = }A)A = (1,00}. To find A 1 we lake 
concatenations of pairs of elements of A, This gives A 3 = {11,100,001,0000}. To find 
A 3 we take concatenations of elements in A 2 and A; this gives A 3 = {111, 1100,1001, 
10000 , 0011 , 00100 , 00001 , 000000 }. ■ 


MflNmON 2. Suppose that A is a subset of V\ 
A, denoted by A*, is the set consisting of 
from A. Thai is, A * « U*„ fl A*. 


EXAMPLE 3 What are the Kleene closures of the sets A = {0}, B - {0,1}, and C = {11}? 

Solution: The Kleene closure of A is the concatenation of the string 0 with itself an 
arbitrary finite number of times. Hence A* = {0 n | n - 0, 1, 2,...}. The Kleene closure 
of B is the concatenation of an arbitrary numberof strings where each string is either Oor 


mb 


Stephen Cole Kleene (I9C9-19M). Stephen Kleene was bom in Hartford, Connecticut. His mother, Alice 
Lena Cole, was a poet, an<l his father, Gustav Adolph Kleene, was an economics professor. Kleene attended 
Amherst College and received his Ph D. from Princeton in 1934, where he studied under the famous logician 
Alonzo Church. Kleene joined the faculty of the University of Wisconsin in 1935, where he remained except 
for several leaves, including stays at the Institute for Advanced Study in Princeton. During World War II he 
was a navigation instructor at the Naval Reserve's Midshipmen's School and later served as the director of 
the Naval Research Laboratory. Kleene made significant contributions to the theory of recursive functions, 
investigating questions of computability and decidability, and proved one of the cemral results of automata 
theory. He served as the Acting Director of the Mathematics Research Center and as Dean of the College of 
Letters and Sciences at the University of Wisconsin, Kleene was a student of natural history. He discovered 
a previously undesenbed variety of butterfly that is named after him. He was an avid hiker and climber. 
Kleene was also noted as a talented teller of anecdotes, using a powerful voice that could be heard several 
offices away. 
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EXAMPLE 4 


1 , This is the set of all strings over the alphabet V = {0,1}, That is, B' - V*. Finally, 
the Kleene closure of C is the concatenation of the string 11 with itself an arbitrary 
number of times. Hence, C* is the set of strings consisting of an even number of Is. 
That is, C* = {1 u j n = 0,1, 2,.. .}. • 

FINITE-STATE AUTOMATA 


We will now give a definition of a finite-state machine with no output. Such machines 
are also called finite-state automata, and that is the terminology we will use for them 
here. (Note: The singular of automata is automaton.) These machines differ from the 
finite-state machines studied in Section 10.2 in that they do not produce output, but they 
do have a set of final states. As we will see, they recognize strings that take the starting 
state to a final state. 


DEFINITION 3. A finite-state automaton M - (S, 1,J, s®, F) consists of a finite 
set S of states, a finite input alphabet l, a transition function / that assigns a next 
state to every pair of state and input, an initial state so, and a subset F of $ conn sting 
fit final states. 


We can represent finite-state automata using either state tables or state diagrams. 
Final states are indicated in state diagrams by using double circles. 


Construct the state diagram for the finite-state automaton M = (5, 1, f, % F), where 
S = {jq, A], s;, S3}, I - { 0 , If, F - {% 53}, and the transition function / is given in 
Table 1. 

Solution: The state diagram is shown in Figure 1. Note that since both the inputs 0 and 
1 take S 2 to s 0 . we write 0,1 over the edge from to so- I 


The transition function/ can be extended so that it is defined for all pairs of states 
and strings. Let x = x\Xj ■ • ■ x* be a string in /* Then /(.5 t , x) is the state obtained by 
using each successive symbol of x, from left to right, as input, starting with state sj. 


TABLE 1 


f 

State 

Input 

0 1 

*0 

■V 0 Sl 

T| 

r® $2 

&2 

.f'n vr, 

-n ,si 



Finite-State Automaton. 
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EXAMPLE 5 


From j] we go on to state s 2 = f(s\, rf), then to state s 2 = /fe< * 2 ). a™ 150 °«> with 
f(si, x) = f(n,x k ). 

A string x is said to be recognized or accepted by the machine M = (5, l, f, % F) 
if it takes the initial state vp to a final state, that is 4 /(% x) is a state in F. The language 
recognized or accepted by the machine M, denoted by U M), is the set of all strings that 
are recognized by M. Two finite-state automata are called equivalent if they recognize 
the same language. 


Determine the language recognized by the finite-state automata M\, Mi, and Mi in 
Figure 2. 

Solution; The only final state of M\ is .sg. The strings that take sq to itself are those 
consisting of zero or more consecutive Is, Hence, L(M{) = {l n | n — 0, 1 , 2,. . .}. 

The only final state of Mi is The only strings that take So to Si are 1 and 01. 
Hence, L(M 2 ) = {1,01}. 

The final states of M 3 are ^ and y,. The only strings that take s 0 to itself are 
A, 0,00, OOO,.... that is, any string of zero or more consecutive 0s, The only strings 
that take so to s 3 arc a string of zero or more consecutive Os, followed by 10, followed 
by any string. Hence, L(AL) = {0", 0"1 Or | n - 0,1, 2,..., and x is any string}. ■ 





M, 

FIGURE 2 Some Finite-State Automata. 
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EXAMPLE 6 


EXAMPLE 7 


The finite-state automata discussed so far are deterministic, since for each pair 
of state and input value there is a unique next state given by the transition function. 
There is another important type of finite-state automaton in which there may be several 
possible next states for each pair of input value and state. Such machines are called 
iiondetemiinislie. Nondeterrainistic finite-state automata are important in determining 
which languages can be recognized by a finite-state automaton. 

DEFINITION 4, A nondeterminislic finite-state automaton M - {S, /./.* o,F) 
consists of a set S of states, an input alphabet /, a transition function / that assigns 
a set of states to each pair of state and input, a starting state ,5 'q, and a subset F of S 
consisting of the final states. 


We can represent nondeterministic finite-slate automata using stale tables or state 
diagrams. When wc use a state table, for each pair of state and input values we give a 
list of possible next states. In the state diagram we include an edge from each state to all 
possible next states, labeling edges with the input or inputs that lead to this transition. 


Find the state diagram for the nondeterministic finite-state automaton with the state 
table shown in Table 2. The final states are vi and ,t 3 . 

Solution: The state diagram for this automaton is shown in Figure 3. ■ 


Find the state table for the nondeterministic finite-state automaton with the state dia¬ 
gram shown in Figure 4. 

Solution: The state table is given as Table 3. ■ 


What does it mean for a nondeterministic finite-state automaton to recognize a 
string x = xj xj - ■ ■ xfi! The first input symbol jp takes the starting state ap to a set 
Si of states. The next input symbol x? takes each of the states in S| to a set of states. Let 


T VBl 1 2 



/ 


State 

0 

Input 

1 


% ■ 

1 

S3 

* 

J'n 


Si. S3 

s': 

__ 

■K). i 

1. si 

% C 

■Si 



FIGURE 3 The Nondetemiioistic Finite-State 
Automaton with State Table Given in Table 2. 
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EXAMPLE 8 


THEOREM 1 


TABLE 3 


L 


State 

Input 

0 I 

so 

% -v 2 


Sl 

S3 

*4 

$2 


S4 

S} 

S3 


Jl _ 

S3 

S3 


o 



FIGURE 4 A Nondeterministic Finite-State Automaton. 


S; be the union of these sets. We continue this process, including at a stage all states 
obtained using a state obtained ai the previous stage and the current input symbol. We 
recognize, or accept, the string v if there is a final state in the set of ail states that can be 
obtained from s,-j using x. The language recognized by a nondeterministic finite-state 
automaton is the set of all strings recognized by this automaton. 


Find the language recognized by the nondeterministic finite-state automaton shown in 
Figure 4. 

Solution: Since s 0 is a final state, and there is a transition from so to itself when 0 is 
the input, the machine recognizes all strings consisting of zero or more consecutive Os. 
Furthermore, since s 4 is a final state, any string that has ,r 4 in the set of states that can be 
reached from sq with this input string is recognized. The only such strings are strings 
consisting of zero or more consecutive Os followed by 01 or 11. Since sq and r 4 are the 
only final states, the language recognized by the machine is {0 n , (TO I, 0" 11 J« ^ 0}. 


One important fact is that a language recognized by a nondeterministic finite-state 
automaton is also recognized by a deterministic finite-state automaton. We will take 
advantage of this fact in the next section when we will determine which languages are 
recognized by finite-state automata. 

If the language L is recognized by a nondeterministic finite-stale automaton Mo , then 
L is also recognized by a deterministic finite-state automaton M\. 

Proof: We will describe how to construct the deterministic finite-state automaton Mi 
that recognizes L from Mt h the nondeterministic finite-state automaton that recognizes 
this language. Each state in M\ will be made up of a set of states in Mo- The start symbol 
of Mi is which is the set containing the start state of Mr,. The input set of M, is 
the same as the input set of M 0 . Given a state {.s,-,,, .17 J of M\. the input symbol 
x takes this state to the union of the sets of next states for the elements of this set, that 
is, the union of the sets f(s , t ), ... /(s ;i )■ The states of M, are all the subsets 

of 5’, the set of states of Mo, that are obtained in this way starting with sq. (There are as 
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FIGURE 5 A Deterministic Automaton Equivalent to the Nondeterministic Automaton 
in Example 7. 


many as 2 " states in the deterministic machine, where n is the number of states in the 
nondeterministic machine, since all subsets may occur as states, including the empty 
set, although usually far fewer states occur.) The final states of M\ are those sets that 
contain a final state of M. 

Suppose that an input string is recognized by Mq. Then one of the states that can 
be reached from .to using this input string is a final state (the reader should provide an 
inductive proof of this). This means that in M \, this input string leads from {sy,} to a set 
of states of M 0 that contains a final state. This subset is a final state of M \, so this string 
is also recognized by M[. Also, an input string not recognized by Ma does not lead to 
any final states in Mq. (The reader should provide the details that prove this statement.) 
Consequently, this input string does not lead from {jq} to a final state in M\. □ 

EXAMPLE 9 Find a deterministic finite-state automaton that recognizes the same language as the 

nondeterministic finite-state automaton in Example 7, 

Solution: The deterministic automaton shown in Figure 5 is constructed from the noil- 
deterministic automaton in Example 7. The states of this deterministic automaton arc 
subsets of the set of all states of the nondeterministic machine. The next state of a subset 
under an input symbol is the subset containing the next states in the nondeterministic 
machine of ail dements in this subset. For instance, on input, of 0, {sq} goes to {% .w}, 
since sj has transitions to itself and to si in the nondeterministic machine; the set {j-q, _v>) 
goes to {x |. s 4 } on input of 1 , since goes just to .iq and sj goes just to ra on input of 1 
in the nondeterministic machine; and the set {iq, j 4 } goes to fo} on input of 0 , since si 
and *4 both go to just rj on input of 0 in the deterministic machine. All subsets diat are 
obtained in this way are included in the deterministic finite-state machine. Note that 
the empty set is one of the states of this machine, since it is the subset containing ail 
the next states of { 1 - 3 } on input of 1. The start state is {x (! }, and the set of final states are 
all those that include ,r 0 or s+, ■ 


Exercises 

1. Let A = {0,11} and B = {00,01}. Find each of the 
following sets. 

a) AB b) BA c) A d) li : 


1. Show that ifA is a set of strings, then A0 - 0A = 0. 
3. Find al 1 pairs uf sets of strings A and B for which A B ■= 
{ 10 , 111 , 1010 , 1000 , 10111 , 101000 }. 
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4. Show that the following equalities hold. 

■)W=W 

b) (A*)* = A* for every set of strings A 

5, Describe the elements of the set A* for the following 
values of A. 

a) {10} b) {III} c) {0,01} d) {1,101} 

6. Lei f'be an alphabet,and letAandfibe subsets of V’*, 
Show that] A B\ - |A||B|. 

7, Let V he an alphabet, and let A and B be subsets of V* 
with A C 6. Show that A*C B*. 

8, Suppose that A is a subset of K* where V is an alphabet. 
Prove or disprove each of the following statements. 

a) A C A 2 b)if A = A 2 , then A £ A c)A{A} = A 
d)(A*)* = A* e)A*A = A* f)|A s | = \A\ h 

9. Determine whether the string 11!01 is in each of the 
following sets. 

a) {o,i}* b) mom* ci {uKirfoi} 

d) {li;*{01}* e) {11I}*{0}*{1} f) {111,000}{00,01} 


10 . Determine whether each of the following strings is rec¬ 
ognized by the deterministic finite-state automaton in 
Figure 1. 

a) 010 b) 1101 c) 1111110 d) 0101010)0 

11 . Determine whether all the strings in each of the follow ¬ 
ing sets are recognized by the deterministic finite-state 
automaton in Figure L 

a) {0}* b) {0}{0}* c) {1} {0}* 

d) {01}* e){0}*{l}* nil HQ I}* 

In Exercises 12-16 find the language recognized by the 
given deterministic finite-stale automaton. 





1 
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In Exercises 17-21 find the language recognized by the 
given nondeterministic finite-state automaton. 



1 


19. 




22. Find a deterministic finite-state automaton that recog¬ 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 17. 

23. Find a deterministic finite-state automaton that recog¬ 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 18. 

24. Find a deterministic finite-state automaton that recog¬ 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 19. 

25. Find a deterministic finite-state automaton that recog¬ 
nizes the same language as the nondeterministic finite- 
state automaton in Exercise 20. 

26. Find a deterministic finite-state automaton that recog¬ 


nizes the same language as the nondeterministic finite- 
stale automaton in Exercise 21. 

27. Find a deterministic finite-state automaton that recog¬ 
nizes each of the following sets, 

a) {0} b) {1,00} 

e) {1" J/t = 2, 3.4_} 

28 , Find a nondeterministic finite-state automaton that 
recognizes each of the languages in Exercise 27, and 
has fewer states, if possible, than the deterministic 
automaton you found in that exercise. 

*29. Show r that there is no finite-state automaton that recog¬ 
nizes the set of bit strings containing an equal number 
of Os and 1 s. 
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104 ____ 

Language Recognition 

INTRODUCTION 


We have seen that finite-state automata can be used as language recognizers. What sets 
can be recognized by these machines? Although this seems like an extremely difficult 
problem, there is a simple characterization of the sets that can be recognized by finite 
state automata. This problem was first solved in 1956 by the American mathematician 
Stephen Kleene. He showed that there is a finite-state automaton which recognizes a set 
if and only if this set can be built up from the null set, the empty string, and singleton 
strings by taking concatenations, unions, and Kleene closures, in arbitrary order. Sets 
that can be built up in this way are called regular sets. 

Regular grammars were defined in Section 10.1, Because of the terminology used, 
it is not surprising that there is a connection between regular sets, which are the sets 
recognized by finite-state automata, and regular grammars. In particular, a set is regular 
if and only if it is generated by a regular grammar. 

Finally, there are sets that cannot be recognized by any finite-state automata. We 
will give an example of such a set. We will briefly discuss more powerful models of 
computation, such as pushdown automata and Titling machines, at the end of this sec¬ 
tion. 

REGULAR SETS 


The regular sets are those that can be formed using the operations of concatenation, 
union, and Kleene closure in arbitrary order, starting with the empty set, the empty 
string, and singleton sets. We will see that the regular sets are those which can be rec¬ 
ognized using a finite-state automaton. To define regular sets we first need to define 
regular expressions. 

DEFINITION1. The regular expressions over a set / are defined recursively by: 

the symbol 0 is a regular expression; 
the symbol A is a regular expression; 
the symbol x is a regular expression whenever x 6 /; 
the symbols (AB)» (A U B), and A* are regular expressions whenever A and B 
are regular expressions. 


Each regular expression represents a set specified by the following rules: 

0 represents the empty set, that is, the set with no strings; 

A represents the set {A}, which is the set containing the empty string; 
x represents the set M containing the string with one symbol x, 

(AB) represents the concatenation of the sets represented by A and by B: 
(A U B) represents the union of the sets represented by A and by B; 

A' represents the Kleene closure of the set represented by A. 
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EXAMPLE 1 


THEOREM 1 


Sets represented by regular expressions are called regular sets. Henceforth regular 
expressions will be used to describe regular sets, so when we refer to the regular set A, 
we will mean the regular set represented by the regular expression A. The following 
example shows how regular expressions are used to specify regular sets. 


What are the strings in the regular sets specified by the regular expressions 10*, (10)*, 
0 U 01, 0(0 U 1)*, and (0*1)*? 

Solution: The regular sets represented by these expressions are given in Tabic 1, as the 
reader should verify. ■ 

KLEENEX THEOREM 


In 1956 Kleene proved that regular sets are the sets which are recognized by a finite- 
state automaton, Consequently, this important result is called KIrene’s theorem. 

KLEENE’S THEOREM A set is regular if and only if it is recognized by a finite- 
state automaton, 

Kleene’s theorem is one of the central results in automata theory'. We will prove the 
only if part of this theorem, namely, that every' regular set is recognized by a finite-state 
automaton. The proof of the t/part, that a set recognized by a finite-state automaton is 
regular, is left as an exercise for the reader. 

Proof: Recall that a regular set is defined in terms of regular expressions, which are 
defined recursively. We can prove that every regular set is recognized by a finite-state 
automaton if we can do the following things, 

1. Show that 0 is recognized by a finite-state automaton. 

2. Show that {A} is recognized by a finite-state automaton, 

.1 Show' that {af is recognized by a finite-state automaton whenever a is a symbol 
in l, 

4 . Show that AB is recognized by a finite-state automaton whenever both A and B 
are. 

5. Show' that A U Sis recognized by a finite-state automaton whenever both A and B 
are, 

6. Show that A* is recognized by a finite-state automaton whenever A is. □ 


TABLE 1 

Expression 

Strings 

10* 

A 1 followed by any number of Os (including no zeros) 

(10)* 

Any number of copies of 1(1 ( including the null string ) 

OuOl 

The string 0 or the string 01 

0(0 U l)* 

Any string beginning with 0 

( 0 *n* 

Any string not ending with 0 
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(a) (b) (c) 


FIGURE 1 Nondeterministic Finite-Stale Automata That Recognize Some Basic Sets. 


We now consider each of these tasks. First, we show that 0 is recognized by a nonde- 
terministic finite-state automaton. To do this, all we need is an automaton with no final 
states. Such an automaton is shown in Figure f(a). 

Second, we show that {A} is recognized by a finite-state automaton. To do this, all 
we need is an automaton that recognizes A, the null string, but not any other string. This 
can be done by making the start state sq a final state and having no transitions so that 
no other string takes in to a final state. The nondeterministic automaton in Figure 1(b) 
shows such a machine. 

Third, we show that {«} is recognized by a nondeterministic finite-state automaton. 
To do this, we can use a machine with a starting state s 0 and a final state s v We have 
a transition from % to jj when the input is a, and no other transitions. The only string 
recognized by this machine is a. This machine is shown in Figure 1(c). 

Next, we show that AB and A U B can be recognized by finite-state automata if A 
and B are languages recognized by finite-state automata. Suppose that A is recognized 
by M a ~ (S A , /, / a , s a , F a ) and B is recognized by M B ~ (S B , I, f B , s B . TV). 

We begin by constructing a finite-state machine M AB = (Sab, F /ab, s as, F ab ) that 
recognizes Ait, the concatenation of A and B. We build such a machine by combining 
the machines for A and B in series, so a string in A takes the combined machine from s A > 
the start state of M A , to s B , the start state ol A/ fi A string in B should take the combined 
machine from sg to a final state of the combined machine. Consequently, we make the 
following construction, Let S AB be 5 A U S b . The starting state s AB is the same as s A . 
The set of final states, Fas, is the set of final states of A/g with s AS included if and 
only if A £ A Cl B. The transitions in M AB include all transitions in M A and in Mg, as 
well as some new transitions. For every' transition in M A that leads to a final state, we 
form a transition in M AB from the same state to on the same input. In this way, a 
string in A takes M A3 from s AB to s St and then a string in B takes s B to a final state of 
Mab- Moreover, for every transition from s R we form a transition in M A b from -tab to 
the same state. Figure 2(a) contains an illustration of this construction. 

We now construct a machine M AuS = {$aub, /, /a us. Saub, Faub) that recognizes 
A U B. This automaton can be constructed by combining U A and Mg in parallel, using 
a new start state that has the transitions that both s A and sb have. Let Saub = S A U 
Sg U (musK where s A uil is a new state that is the start state of M AuB . Let the set of final 
states Fa u a be f ' A UFgU {jaus! if A £ A U B, and F A U Fg otherwise. The transitions 
in A/au b include all those in M A and in Mg. Also, for each transition from s A to a state j 
on input i we include a transition from Jaus to s on input t, and for each transition from 
rg to a state s on input i we include a transition from y AU b to s on input i. In this way, a 
string in A leads from s'aub to a final state in the new machine, and a string in B leads 
from ,s MJB to a final state in the new machine. Figure 2(b) illustrates the construction 

of M aub . 

Finally, we construct M A . = (S r , I, f A .,s A ; F A -), a machine that recognizes A*, 
the Kleene closure of A. Let S A . include all states in 5 a and one additional state v v > 
which is the starting state for the new machine. The set of final states F A - includes all 
states in F A as well as the start state s A . , since A must be recognized. To recognize 
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fa) 


Tmsitinn to final slate in .W^ produces a transition to 



Tran sir ion from .v^ in Aproduces a iran-iiion from - .v^ 


Sr . : jj sNjSc j- ' lp - •=■■ '... a':uc'! is 1 ’ii.ll; it ami s,. arc final. 


Final states include aii finite states uf M$. 



,; L ’ > ']' i\i r i s i 1111 n s f r o iv. a prod u ce A ira n s i [ i on s 1'rc «n t, an J a.M final % tales of Af. . 



•, . j., i he r.c'A sTLLi-1 s[:l'c ^hish U *i final suttc. Final stuTes include all final states in M.\. 

FI(a KK 2 Building Automata to Recognize Concatenations, Unions, and Kleene Closures. 


concatenations of arbitrarily many strings from ,4, we include all the transitions in M 4 , 
as well as transitions from ,s A . that match the transitions from .s,*, and transitions from 
each final state that match the transitions from s A . With this set of transitions, a string 
made up of concatenations of strings from A will take y, r to a final state when the first 
string in A has been read, returning to a final state when the second string in ,4 has been 
read, and so on. Figure 2(c) illustrates the construction we used. 

A nondetermimstic finite-state automaton can be constructed for any regular set 
using the procedure described in this proof, We illustrate how this is done with the 
following example. 


UX AMFI.F, 2 Construct a nondetermimstic finite-state automaton that recognizes the regular set 

ruoi. 
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THEOREM 2 


EXAMPLE 3 


Solution: We begin by building a machine that recognizes 1*. This is done using the 
machine that recognizes 1 and then using the construction for M v described in the 
proof. Next, we build a machine that recognizes 01, using machines that recognize 0 
and 1 and the construction in the proof for M A b• Finally, using the construction in the 
proof for A/,iua, we construct the machine for 1* U 01, The finite-state automata used in 
this construction are shown in Figure 3. The states in the successive machines have been 
labeled using different subscripts, even when a state is formed from one previously used 
in another machine. Note that the construction given here does not produce the simplest 
machine that recognizes 1* U 01, A much simpler machine that recognizes this set is 
shown in Figure 3(b), ■ 

REGULAR SETS AND REGULAR GRAMMARS 

In Section 10.1 we introduced phrase-structure grammars and defined different types of 
grammars. In particular we defined regular, or type 3, grammars, which are grammars 
of the form G = (V, T, S, P) where each production is of the form S —» A, A —» a, 
or A -*■ aB. where a is a terminal symbol, and A and B are nonterminal symbols. As 
the terminology suggests, there is a close connection between regular grammars and 
regular sets. 

A set is generated by a regular grammar if and only if it is a regular set. 

Proof: First we show that a set generated by a regular grammar is a regular set. Suppose 
that G - {V, T, S, P ) is a regular grammar generating the set L(C). To show that LiG) 
is regular we will build a nondeterministic finite-state machine M = {5, /, f, % F) that 
recognizes L(G). Let 5, the set of states, contain a state for each nonterminal symbol 
,4 of G and an additional state s?, which is a final stale. The start state sq is the state 
formed from the start symbol S, The transitions of M are formed from the productions 
of G in the following way. A transition from to s/. on input of a is included if A —* a 
is a production, and a transition from to sg on input of a is included if .4 -» aB 
is a production. The set of final states includes Sf and also includes j 0 if S — A is a 
production in G. It is not hard to show that the language recognized by M equals the 
language generated by the grammar G, that is, the L(M) = L(G). This can be done by 
determining the words that lead to a final state. The details are left as an exercise for 
the reader. □ 

Before giving the proof of the converse, we illustrate how a nondeterministic ma¬ 
chine is constructed that recognizes the same set as a regular grammar. 


Construct a nondeterministic finite-state automaton that recognizes the language gener¬ 
ated by the regular grammar G = (K T. 5, P), where V = {0,3, .4,5}, T - {0, 1}, and 
the productions in Pare S —> 14, S 0, S -» A, 4 —* 04. A -* 14, and 4 -* 1 . 

Solution: The state diagram for a nondeterministic finite-slate automaton that recog¬ 
nizes LiG) is shown in Figure 4. This automaton is constructed following the procedure 
described in the proof. In this automaton Sq is the stale corresponding to S, is the state 
corresponding to 4, and is the final state. ■ 


We now complete the proof of Theorem 2. 
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FlCt'RE 3 Nondeterministk Finite-State Automata Recognizing 1 ‘ U 01. 
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EXAMPLE 4 



FIGURE 4 A Nondeterministic Finite-State Automaton Recognizing L(G), 


Proof: We now show that if a set is regular, then there is a regular grammar that gen¬ 
erates it. Suppose that M is a finite-state machine which recognizes this set with the 
property that % the starting state of M, is never the next state for a transition. ( We can 
find such a machine by Exercise 14.) The language G = ( VT, S, P) is defined as fol¬ 
lows. The set V of symbols of G is formed by assigning a symbol to each state of 5 and 
each input symbol in I. The set T of terminal symbols of G is the symbols of G formed 
from the input symbols in /. The start symbol S is the symbol formed from the start state 
.to- The set P of productions in G is formed from the transitions in M, In particular, if 
the state s goes to a final state under inpul a, then the production A, — a is included 
in P, where A s is the nonterminal symbol formed from the state s. If the state s goes to 
the state t on input a, then the production A s — a A, is included in P. The production 
S —* A is included in P if and only if A E L(M). Since the productions of G correspond 
to the transitions of M and the productions leading to terminals correspond to transi¬ 
tions to final states, it is not hard to show that L(G) - L{M). We leave the details as an 
exercise for the reader, □ 

The following example illustrates the construction used to produce a grammar from 
an automaton that generates the language recognized by this automaton. 


Find a regular grammar that generates the regular set recognized by the finite-state 
automaton shown in Figure 5. 



FIGURE 5 A Finite-State Automaton. 
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Solution: The grammar G - (K T S, P } generates the set recognized by this automa¬ 
ton where G = {5. A. B. 0, 1}; where the symbols S, A, and B correspond to the states 
% a'}, and S 2 , respectively; T = {0,1}; S is the start symbol; and the productions are 
S 0A,S -> 15,5 -> l,S-> A, A ^0A, A -+ IB, A -■ 1.B-+ 0 A, B -* 15, 
and 5 — 1. ■ 


A SET NOT RECOGNIZED BY A FINITE-STATE AUTOMATON 


We have seen that a set is recognized by a finite-state automaton if and only if it is 
regular. We will now show that there are .sets which are not regular by describing one 
such set. The technique used to show that this set is not. regular illustrates an important 
method for showing that certain sets are not regular. 


EXAMPLE 5 Show that the set {G" 1" | n = 0,1, 2,...}, made up of all strings consisting of a block 

of Os followed by a block of an equal number ofls, is not regular. 

Solution: Suppose that this set were regular. Then there would be a deterministic finite- 
state automaton M = (S, l, f, s fh F) recognizing it. Let .Vbe the number of states in this 
machine, that is, JV = |$|. Since M recognizes all strings made up of a number of Os 
followed by an equal number of Is, M must recognize 0^1*. Let % s h s 2 , ■ . ,,S 2 ,\ 
be the sequence of states which is obtained starting at sa and using the symbols 

of 0*1" as input so that j, = f(s 0 , 0), s 2 = /<ji,0 )_% = f(s K ~ 1 , 0).% +1 = 

/(s.v, 1)__ sjs = f{siN- 1 . U- Note that s 2 n is a final state. 

Since there are only N states, the pigeonhole principle shows that at least two of the 

first N + I of the states, which are .to_, j#, must be the same. Say that s, and t / are 

two such identical states, with 0 < i < j s Afi This means that f(s[. O') - sj where 
f = i - j, It follows that there is a loop leading from j, back to itself, obtained using 
the input 0 a total of 1 times, in the state diagram shown in Figure 6. 

Now consider the input string O v CK I N - O w+ ' t A \ There are / more consecutive Os 
at the start of this block than there are consecutive Is that follow it. Since this string 
is not of the form O' 1 1" (since it has more Os that Is), it is not recognized by M, Con¬ 
sequently, /(% T w, ]' v ) cannot be a final state. However, when we use the string 
Q,V + / ] .v 
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as input, we end up in the same state as before, namely, J 2 JV- The reason for 
this is that the extra f Os in this string take us around the loop from back to itself 
an extra time, as shown in Figure 6. Then the rest of the string leads us to exactly 
the same state as before. This contradiction shows that {0 n \ n \ tt = 1,2,...} is not 
regular. ■ 


MORE POWERFUL TYPES OF MACHINES 


Finite-state automata are unable to carry out many computations. The main limitation of 
these machines is their finite amount of memory. This prevents them from recognizing 
languages that are not regular, such as {0” 1 n \ n - 0,1, 2,...}. Since a set is regular if 
and only if it is the language generated by a regular grammar, Example 5 shows that 
there is no regular grammar which generates the set {O'T In - 0, 1, 2,...}, However, 
there is a context-free grammar which recognizes this set. Such a grammar was given 
in Example 5 in Section 10.1. 

Because of the limitations of finite-state machines, it is necessary to use other, more 
powerful, models of computation. One such model is the pushdown automaton, A 
pushdown automaton includes everything in a finite-state automaton, as well as a stack, 
which provides unlimited memory. Symbols can be placed on the top or taken off the 
top of the stack. A set is recognized in one of two ways by a pushdown automaton. 
First, a set is recognized if the set consists of all the strings that produce an empty stack 
when they are used as input. Second, a set is recognized if it consists of all the strings 
that lead to a final state when used as input. It can be shown that a set is recognized 
by a pushdown automaton if and only if it is the language generated by a context-free 
grammar. 

However, there are sets that cannot be expressed as the language generated by 
a context-free grammar. One such set is {(VI"2" | n = 0, 1,2,...}. We will indi¬ 
cate why this set cannot be recognized by a pushdown automaton, but we will not 
give a proof, since we have not developed the machinery needed, (However, one 
method of proof is given in Exercise 28 of the supplementary exercises at the end 
of this chapter.) The stack can be used to show that a string begins with a sequence 
of 0 s followed by an equal number of Is by placing a symbol on the stack for each 
0 (as long as only Os are read), and removing one of these symbols for each 1 (as 
long as only Is following the 0s are read). But once this is done, the stack is empty, 
and there is no way to determine that there are she same number of 2 s in the string 
as 0 s. 

There are other machines called linear hounded automata, more powerful than 

pushdown automata, that can recognize sets such as {G'T2 f! | n = 0,1,2_}. In 

particular, linear bounded automata can recognize context-sensitive languages. How ¬ 
ever, these machines cannot recognize all the languages generated by phrase-structure 
grammars. To avoid the limitations of special types of machines, the model known 
as a Turing machine, named after the British mathematician Alan Turing, is used. 
A Turing machine is made up of everything included in a finite-state machine to¬ 
gether with a tape, which is infinite in both directions. A Turing machine has read 
and write capabilities on the tape, and it can move back and forth along this tape. 
Turing machines can recognize all languages generated by phrase-structure gram¬ 
mars, In addition, Turing machines can mode] all the computations that can be per¬ 
formed on a computing machine. Because of their power, Turing machines are exten¬ 
sively studied in theoretical computer science. We will briefly study them in the next 
section. 
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Exercises 


1. Describe in words the strings in each of the foliowing 
regular sets, 

a) 1*0 b) 1*00* 

ci in u ooi d) (i u oo)* 

e) (00*1)* f) (OU 1)(0U 1}*00 

2. Determine whether 1011 belongs to each of the follow¬ 
ing regular sets. 

a) 10*1* b) 0*(10 U 11)* 

c) 1(01 )*1* d) 1*01(0 U I) 

e) (10)*(11)* f) 1(00)*(11)* 

g) (10)*I011 It ) (1 U 00-.01 U 0)1* 

X Express each of the following sets using a regular ex¬ 
pression. 

a) the set of strings of one or more Os followed by a 1 
hi the set of strings of two or more symbols followed 
bv three or more Os 

c) the set of strings with either no I preceding a 0 or 
no 0 preceding a l 

d) the set of strings containing a string of Is so that 
the number of Is equals 2 modulo 3, followed by 
an even number of Os 


4, Construct deterministic finite-state automata that rec¬ 
ognize the following sets from where l is an alpha¬ 
bet. 

a) 0 b) {A} c) {«}, where a £ l 
*5. Show that if A is a regular set, then A R , the set of all 
reversals of smogs in A, is also regular. 


6. Find a finite-state automaton that recognizes 
a) {A,0}. b) {0,11}. c) {0,11,000}. 

7. Using the constructions described in the proof of 
Kleene’s theorem, find nondetenninistic finite-state au¬ 
tomata that recognize each of the following sets. 

a) 0*1* b) <0 u 11)* c) 01* U 00*1 

8. Construct a nondetenninistic finite-state automaton 
that recognizes the language generated by the regular 
grammar G = (V, T, S, P) where V = {0,1, S, A, 5}, 
T = {0,1}, $ is the start symbol, and the set of pro¬ 
ductions is 

a) S - OA, S -* IB, A —> Q, B —> 0. 

b) 5 -> LA, S — 0, S A, A — OS, B IS, 
B -* 1. 

e) S - IS, S -*■ 0, A — 1A, A — OS, A -» 1, 
A — 0, B - 1. 

In Exercises 9-11 construct a regular grammar G = 

(V, r, S. P) that generates the language recognized by the 
given finite-state machine. 



Alan Math (son luring (1912-1954). Alan Turing was bom in London, although he was conceived in 
India, where his father was employed in the Indian. Civil Service. As a boy, he was fascinated by chem¬ 
istry, performing a wide variety of experiments, and by machinery, Turing attended Sherborne, an English 
boarding school In 1931 he won a scholarship to King’s College, Cambridge, After completing his dis¬ 
sertation, which included a rediscover)' of the central limit theorem, a famous theorem in statistics, he 
was elected a fellow of his college. In 1935 Turing became fascinated with the decision problem, a prob¬ 
lem posed by the great German mathematician Hilbert, which asked whether there is a general method 
that can be applied to any assertion to determine whether the assertion is true, Turing enjoyed running 
(later in life running as a serious amateur in competitions), and one day, while resting after a run, be 
discovered the key ideas needed to solve the decision problem. In his soMod, he invented what is now 
called a Turing machine as the most general model of a computing machine. Using these machines, he 
found a problem, involving what he called computable numbers, that could not be decided using a genera! 
method. 

From 1936 to 1938 Turing visited Princeton University to work with Alonzo Church, who had also 
solved Hilbert’s decision problem. In 1939 Turing returned to King’s College. However, at the outbreak of 
Worid War H, he joined the Foreign Office, performing cryptanalysis of German ciphers. His contribution 
io the breaking the code of the Enigma, a mechanical German cipher machine, played an important role in 
winning the war, 

A fter the war, Turing worked on the development of early computers. He was interested in the ability of 
machines to think, proposing that if a computer could not be distinguished from a person based on written 
replies to questions, it should be considered to be “thinking.’’ He was also interested in biology, having 
written on morphogenesis, the development of form in organisms. In 1954 Turing committed suicide by 
taking cyanide, wiihout leaving a clear explanation. Legal troubles related to a homosexual relationship and 
hormonal treatments mandated by the court to lessen his sex drive may have been factors in his decision to 
end his life. 
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generates the set recognized by this automaton. 

14. Show that every nondeterministic finite-state automa¬ 
ton is equivalent to another such automaton that has the 
property that its starting state is never revisited. 

*15. Let M = (5, /, /, % F) be a deterministic finite-state 
automaton. Show that the language recognized by 
M, UM\ is infinite if and only if there is a word r 
recognized by M with !(x) a; jS|, 

*16. One important technique used to prove that certain sets 
are not regular is the pumping lemma. The pumping 
lemma states that if M = (S, /, /, % F) is a determin¬ 
istic finite-state automaton and if a is a string in L(M), 
the language recognized by A-f, with l(x) 3 |5|, 
then there are strings a, v, and w in F such that 
x = uvw, l(uv) ^ |5| and /(v) a 1, and uv'w e LiM) 
for i = 0, 1,2,— Prove the pumping lemma. (Hint: 
Use the same idea as was used in Example 5.) 

*17. Show that the set -fO 2 * 1"J- is not regular. You may use 
the pumping lemma given in Exercise 16. 

*18. Show that the set {l' [1 n = 0,1,2,...} is not regular. 

You may use the pumping lemma given in Exercise 16. 
*19. Show that the set of palindromes over {0,1} is not regu¬ 
lar. You may use the pumping lemma given in Exercise 
16. (Hint Consider strings of the form Q K 1O v ,) 

**20. Show that a set recognized by a finite-state automaton 
is regular. (This is the if pm of Kleene’s theorem.) 

10.5 

Turing Machines 

INTRODUCTION 


The finite-state automata studied earlier in this chapter cannot be used as general models 
Web ot computation. They are limited in what they can do. For example, finite-state automata 
are able to recognize regular sets, but not able to recognize many easy-to-deseribe sets, 
including {0 n l n | n 3: €}, which computers recognize using memory. We can use finite- 
state automata to compute relatively simple functions such as the sum of two numbers, 
but we cannot use them to compute functions that computers can, such as the product 
of two numbers. To overcome these deficiencies we can use a more powerful type of 
machine known as a Turing machine, after Alan Turing, the famous mathematician and 
computer scientist who invented them in the 1930s. 

Basically, a Turing machine consists of a control unit, which at any step is in one of 
finitely many different states, together with a tape divided into cells, which is infinite 
in both directions. Turing machines have read and write capabilities on the tape as 
the control unit moves back and forth along this tape, changing states depending on 
the tape symbol read. Turing machines are more powerful than finite-state machines 
because they include memory capabilities that finite-state machines lack. We will show 
how to use Turing machines to recognize sets, including sets that cannot be recognized 
by finite-state machines. We will also show how to compute functions using Turing 
machines, Turing machines are the most genera) models of computation; essentially, 
they can do whatever a computer can do. 



12. Show' that the finite-state automaton constructed from a 
regular grammar in the proof of Theorem 2 recognizes 
the set generated by this grammar. 

13, Show that the regular grammar constructed from 
a finite-state automaton in the proof of Theorem 2 
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DEFINITION OF TURING MACHINES 

Wc now give the formal definition of a Tun rig machine. Afterward we will explain how 
this formal definition can be interpreted in terms of a control head which can read and 
write symbols on a tape and move either right or left along the tape. 


DEFINITION l, A Turing machine T ~ (5, l, f, $o) consists of a finite set S of 
states, an alphabet / containing the blank symbol B t a partial function/ from S x / 
to 5 X / x {/f, L\, and a starting state % 


Recall from the preamble to Exercise 39 in Section 1.6 that a partial function 
is defined only for those elements in its domain of definition. This means that for 
some (state, symbol) pairs the partial function f may be undefined, but for a pair for 
which it is defined, there is a unique (state, symbol, direction) triple associated to 
this pair. 

To interpret this definition in terms of a machine, consider a control unit and a tape 
divided into cells, infinite in both directions, having only a finite number of nonblank 
symbols on it at any given time, as pictured in Figure 1. The action of the Turing ma¬ 
chine at each step of its operation depends on the value of the partial function / for the 
current state and tape symbol. 

At each step, the control unit reads thecunrenl tape symbol x. If the control unit is in 
state ,5 and if the partial function/ is defined for the pair (,t, x) with f(s, x) = (s', x\d). 
the control unit: 

/. enters the state s', 

2. writes the symbol .t' in the current cell, erasing .v, and 

1 moves right one cell if = R or moves left one cell if d = L. 

We write this step as the five-tuple (s, x. s', x\d). If the partial function f is undefined 
for the pair (s, .r), then the Turing machine T will halt. 

A common way to define a Turing machine is to specify a set of five-tuples of the 
form (j, x, s', x\ d). The set of slates and input alphabet is implicitly defined when such 
a definition is used. 

At the beginning of its operation a Turing machine is assumed to be in the initial 
state s 0 and to be positioned over the leftmost nonblank symbol on the tape. If the tape is 
all blank, the control head can be positioned over any cell. We will call the positioning 
of the. control head over the leftmost nonblank tape symbol the initial position of the 
machine. 


Control 




Unit " 

' V’ 1 




l 

-»— ReaJ/Wriie HeaJ 


I 1 1 p 

"■ H ; B 1 1 

II 

j! 

t | /f j o 

1 ' 1 ' 1 “ 


Tape r.s infinite in both directions. 


Only finite many nonhljrik edh at an> time. 

FIGURE 1 4 Representation of a Hiring Machine. 
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EXAMPLE 1 


The following example illustrates how a Turing machine works. 


What is the final tape when the Turing machine T defined by the seven five-tuples 
(i-o, 0. % O.R), (% 1, *!. 1, R ), fan, B. sj, B, R), ( fl , 0, j 0l 0, R ), fai, 1 , s 2 , ft L), 

fai, B, s\, B, R), faz, 1, Si, 0, R) is run on the tape shown in Figure 2(a)? 

Solution: W'e start the operation with T in stale sq and with T positioned over the left¬ 
most nonblank symbol on the tape. The first step, using the five-tuple fao, 0, %, 0, R), 
reads Lhe 0 in the leftmost nonblank cell, stays in state .sq, writes a 0 in this cell, and 
moves one cell right. The second step, using the five-tuple {% 1, si, 1, R), reads the 1 
in the current cell, enters state -c, writes a 1 in this cell, and moves one cell right. The 
third step, using the five-tuple fa]. 0 , % 0 , R), reads the 0 in the current cell, enters state 





_Machine Halts 

FIGURE 2 The Steps Produced by Running 7 on the Tape in Figure 1. 
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EXAMPLE 2 


% writes a 0 in this cell, and moves one cell right. The fourth step, using the five¬ 
tuple (jo- 1, J|. h /f), reads the 1 in the current cell, enters state j|. writes a I in this 
cel!, and moves right one cell. The fifth step, using the five-tuple fy, 1, y, 0, £.). 
reads the 1 in the curretit cell, enters state Sj, writes a 0 in this cell, and moves 
left one cell. The sixth step, using the five-tuple (% 1, ji. 0, R), reads the 1 in the 
current cell, enters the state y, writes a 0 in this cell, and moves right one cell. 
Finally, in the seventh step, the machine halts because there is no five-tuple begin¬ 
ning with the pair (j 3 ,0) in the descnpticm of the machine. The steps are shown in 
Figure 2. 

Note that T changes the first pair of consecutive Is on the tape to Os and then halts 


USING TURING MACHINES TO RECOGNIZE SETS 

Turing machines can be used to recognize sets. To do so requires that we define the 
concept of a final state as follows. A final state of a Turing machine Tis a state that is 
not the first state in any five-tuple in the description of T using five-tuples (for example, 
state i t in Example 1). 

We can now' define what it means for a Turing machine to recognize a string Given 
a string, vve write consecutive symbols in this string in consecutive cells. 


DEFINITION 2, Let V he a subset of an alphabet T. A Turing machine T - 
(S, /, /, ip) recognizes a string x in V* if and only if T, slatting in the initial position 
when x is written on the tape, baits in afinal state. 7 is said to recognize a subset A 
of F* if x is recognized by T if and only if x belongs to A. 


Note that to recognize a subset A of V rt we can use symbols not in V. This means 
thai the input alphabet / may include symbols not in V'. These extra symbols are often 
used as markers (see Example 3). 

When does a Turing machine T not recognize a siring Jt in V*2 The answer is that a 
is not recognized if 7 does not halt or halts in a state that is not final when it operates on 
a tape containing the symbols of x in consecutive cells, starting in the initial position. 
(The reader should understand that this is one of many possible ways to define how to 
recognize sets using Turing machines. ) 

We illustrate this concept with the following example. 


Lind a Turing machine that recognizes the set of hit strings which have a I as their 
second bit (that is, the regular set (0 y 1)1(0 y I f }. 

Solution: We want a Turing machine that, starting at the leftmost nonblank tape cel], 
moves right, and determines whether the second symbol is a 1. If the second symbol 
is 1, the machine should move into a final state. If the second symbol is not a !, the 
machine should not halt or it should halt in a nonfinal state. 

To construct such a machine, we include the five-tuples (%,0,y, 0,/f) and 
Go, 1, y, l, R) to read in the first symbol and put the Turing machine in state s\. 
Next, we include the five-tuples (s L ,0, s 7 , 0, R) and {s x . 1. s s , 1, R) to read in the second 
symbol and either move to state y if this symbol is a 0, or to state ,y if this symbol is 
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EXAMPLE 3 


a 1. We do not want to recognize strings that have a 0 as their second bit, so $2 should 
not be a final state. We want to be a final state. So, we can include the five-tuple 
($ 2 ,0, S 2 p 0, R). Since we do not want to recognize the empty string nor a string with one 
bit, we also include the five-tuples (% B, sj, 0, R) and (s\, B, *zA R). 

The Turing machine T consisting of the seven five-tuples listed above will termi¬ 
nate in the final state a t if and only if the bit string has at least two bits and the second 
bit of the input string is a 1. If the bit string contains fewer than two bits or if the second 
bit is not a !, the machine will terminate in the nonfinal state .v 2 . ■ 


Given a regular set, a Turing machine that always moves to the right can be built 
to recognize this set (as in Example 2), To build the Turing machine, first find a finite- 
state automaton that recognizes the set and then construct a Turing machine, using the 
transition function of the finite-state machine, always moving to the right. 

We will now show how to build a Turing machine that recognizes a nonregular set. 


Find a Turing machine that recognizes the set {0 n I “ | n > 1}. 

Solution: To build such a machine, we will use an auxiliary tape symbol M as a marker. 
We have V = {0, 1} and I = {0,1, A/}. We wish to recognize only strings in V*. We will 
have one final state, The Turing machine successively replaces a 0 at the leftmost 
position of the string with an Af and a i at the rightmost position of the string with an 
M, sweeping back and forth, terminating in a final state if and only if the string consists 
of a block of 0 s followed by a block of the same number of Is. 

Although this is easy to describe and is easily carried out by a Turing machine, 
the machine we need to use is somew hat complicated. We use the marker M to 
keep track of the leftmost and rightmost symbols we have already examined. The 
five-tuples we use are; (% 0, s u M, R), (jj, 0,S|, 0, R), (a,, 1, 1. /?), (a,, M, s 7 , M, L). 

(■-b ri. R, LX (a:, 1, A 3 , Af, L), (S 3 . 1, A 3 , 1, L), (S 3 ,0, S 4 , 0, L), (ss, M, A 5 , M, R), 
(j’ 4 ,0, S 4 , 0, L), (s 4 , M, S 9 , M, R), (a 5 , M, % Af, R). For example, the string 000111 would 
successively become A/00111, M 00llAf, AfA/OllAf, MM0XMM, MMM1MM, 
MMMMMM as the machine operates until it halts. (Note that this string is not changed 
by all steps of the operation of the Turing machine,} 

We leave it to the reader (Exercise 13 at the end of this section) to explain the 
actions of this Turing machine and to explain why it recognizes the set {O'* 1^ j n > 1}. 


It can be shown that a set can be recognized by a Turing machine if and only if it 
can be generated by a type 0 grammar, or in other words, if the set is generated by a 
phrase-structure grammar. The proof will not be presented here. 


COMPUTING FUNCTIONS WITH TURING MACHINES 

A Turing machine can be thought of as a computer that finds the values of a partial 
function. To see this, suppose that the Turing machine T, when given the string x as 
input, halts with the string y on its tape. We can then define T(x) = y. The domain of 
T is the set of strings for which T halts; T(x) is undefined if T does not halt when given 
x as input. Thinking of a Turing machine as a machine that computes the values of a 
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function on strings is useful, but how can we use Turing machines to compute functions 
defined on integers, on pairs of integers, on triples of integers, and so on? 

To consider a Turing machine as a computer of functions from the set of it-tuples 
of nonnegative integers to the set of nonnegative integers (such functions are called 
number-theoretic functions), we need a w ay to represent £-tuples of integers on a 
tape. To do so, we use unary representations of integers. We represent the nonnegative 
integer n by a string of n + 1 Is so that, for instance, 0 is represented by the string i and 

5 is represented by the string 111 111. To represent the Aotuple (n u n 2 ./?*), we use a 

string of n, + 1 Is, followed by an asterisk, followed by a string of n 2 + 1 1 s, followed 
by an asterisk, and so on, ending with a string of n k -Mis. For example, to represent 
the four-tuple (2,0, 1,3} we use the string 111 + 1*11*1111. 

We can now consider a Turing machine T as computing a sequence of number- 

theoretic functions T, T 2 . T k .The function T k is defined by the action of T 

on ^-tuples of integers represented by unary representations of integers separated by 
asterisks. 


EXAMPLE 4 Construct a Turing machine fur adding two nonnegative integers. 

Solution: We need to build a Turing machine T that computes the function f{n ]t n 2 ) = 
m + ni. The pair C«]. n 2 ) is represented by a string of u + 1 Is followed by an asterisk 
followed by m + 1 Is. The machine T should take this as input and produce as output 
a tape w'ith ri] ■+ n% + 1 ones. One way to do this is as follows. The machine starts 
at the leftmost 1 of the input string, and carries out steps to erase this 1, halting if 
Hi = 0 so that there are no more Is before the asterisk, replaces the asterisk with the 
leftmost remaining 1, and then halts. We can use the following live-tuples to do this: 
t% Ui. B. R), U,,*, si, B, R), (a,. U 2 . B, R). (s 2 , 1, s 2 , I, R), (s h *. 1, R). ■ 


Unfortunately, constructing Turing machines to compute relatively simple func¬ 
tions can be extremely demanding. For example, one Turing machine for multiplying 
tw'o nonnegative integers found in many books has 31 five-tuples and 11 states. If it is 
challenging to construct Turing machines to compute even relatively simple functions, 
what hope do we have of building Turing machines for more complicated functions? 
One way to simplify this problem is to use a multitape Turing machine that uses more 
than one tape simultaneously and to build up multitape Turing machines for the com¬ 
position of functions. It can be shown that for any multitape Turing machine there is a 
one-tape Turing machine that cun do the same thing. 

A function that can be computed by a Turing machine is called computable. It is 
web foMy straightforward to show that there are number-theoretic functions which are not 
computable. However, it is not so easy to actually produce such a function. The him 
beaver function defined in the preamble to Exercise 23 at the end of this section is an 
example of a noncomputabie function. One way to show that the busy beaver function 
is not computable is to show that it grows faster than any computable function. (See 
Exercise 24.) 

DIFFERENT TYPES OF TURING MACHINES 

There are many variations on the definition of a Turing machine. We can expand the 
capabilities of a Turing machine in a wide variety of ways. For example, we can allow a 
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Turing machine to move right, left, or not at all at each step. We can allow a Turing ma¬ 
chine to operate on multiple tapes, using (2 + 3n)-tuples to describe the Turing machine 
when n tapes are used. We can allow the tape to be two-dimensional, where at each step 
we move up, down, right, or left, not just right or left as we do on a one-dimensional 
tape, We can allow multiple tape heads that read different cells simultaneously, Further¬ 
more, we can allow a Turing machine to be nondeterministic, by allowing a (state, tape 
symbol) pair to possibly appear as the first elements in more than one five-tuple of the 
Turing machine. We can also reduce the capabilities of a Turing machine in different 
ways. For example, we can restrict the tape to be infinite in only one dimension or we 
can restrict the tape alphabet to have only two symbols. All these variations of Turing 
machines have been studied in detail. 

The crucial point is that no matter which of these variations we use, or even which 
combination of variations we use, we never increase or decrease the power of the ma¬ 
chine. Anything that one of these variations can do can be done by the Turing machine 
defined in this section, and vice versa. The reason that these variations are useful is that 
sometimes they make doing some particular job much easier than if the Turing machine 
defined in Definition 1 were used. They never extend the capability of the machine. 

THE CHURCH-TURING THESIS 

Turing machines are relatively simple. They can have only finitely many states and they 
can read and write only one symbol at a time on a one-dimensional tape. But it turns out 
that Turing machines are extremely powerful. We have seen that Turing machines can 
be built to add numbers and to multiply numbers. Although it may be difficult to actually 
construct a Turing machine to compute a particular function that can be computed with 
an algorithm, such a Turing machine can always be found. This was the original goal 
of Turing when he invented his machines. 

Furthermore, there is a tremendous amount of evidence for the C’hurch-Turing 
thesis, which states that given any problem which can be solved with an effective algo¬ 
rithm, there is a Turing machine that can solve this problem. The reason this is called a 
thesis rather than a theorem is that the concept of solvability by an effective algorithm 
is informal and imprecise, as opposed to the notion of solvability by a Turing machine, 
which is formal and precise. Certainly, though, any problem that can be solved using a 
computer with a program written in any language, perhaps using an unlimited amount 
of memory, should be considered effectively solvable. 

Many different formal theories have been developed to capture the notion of ef¬ 
fective computability. These include Taring’s theory and Church’s lambda-calculus, 
as well as theories proposed by Kleene and by Post. These theories seem quite dif¬ 
ferent on the surface. The surprising thing is that they can be shown to be equivalent 
by demonstrating that they define exactly the same class of functions. With this evi¬ 
dence, it seems that Turing’s original ideas, formulated before the invention of modem 
computers, describe the ultimate capabilities of these machines. 


Alnnsro Church (.1903—19»5i. Alonzo Church was bum in Washington, D.C He studied at Gottingen 
Wfh uniler H '*' 3ert an ^ ' attr ' n Amsterdam. Ho was u member of the faculty at Princeton University from 1927 
until 1967 when he moved to UCLA. Church was one of the founding members of the Association for Sym¬ 
bolic Logie. He made many substantial contributions to the theory of computability, including his solution 
to the decision problem, his invention of the lambda-calculus, and, of course, his statement of what is now 
known as the Church-luring thesis. Among Church’s students were Stephen Kleene and Alan Turing. He 
published articles past his 9()th hinhduy. 
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Exercises 


1. Let T be the Turing machine defined by the five- 
tuples: (% (),x,, I, R), (y ti , 1,%0, ft), fen, BO'i.O.fi), 
til, 0, jj, 1. L), (*,. 1, ,t|. 0, R). (.!’[, B, .t;, 0, L). For each 
of the Following initial tapes, determine the final tape 
when T halts, assuming that T begins in initial position. 


a) 

■■■ B 

e ; 0 

K 

i i 

B 

B 


b) 








-13 

b 1 

0 ! 

[III] 

R I 

B 



c) 


^rn 

1 1 

B | 0 j 

n 

1 










d) 

-1 B i 

B i M | 

ti I 

R | H | 

jlJ 

B 



2. Let I be the Turing machine defined by the five- 
tuples: (,%0, jfi.O, R), (% I, i, (). Z.I. U( t ,B, .s L . 1, R). 

1 , 1 . .T|, 1 , R\ ($ u B,Si, 0 , R),Ui. B,Si, 

0, R), For each of the following initial tapes, determine 
the final tape when T halts, assuming that T begins in 
initial position. 



b) -~j~ ft |" R I 1' | I | l | S | 8 | II 


Cl -■ j fl | B | t) | 0 I B | t) ' 0 | g | ~ 



3. VVhat does the Turing machine described by (he five- 
tuples (% 0. .t B , 0, R), {% 1, -v i, 0, ft i, (jq. 5. .tv, B, R), 
f.vi, 0.4i, 0, R) : Ur L 1, R), and U B,s z ,B,R) do 
when given a hit string as input? 

4-. What does the Turing machine described by the fi ve- 
tuples (jq,0,.V], B. R). I.si, [. R), (.(|.0, i s .O, ft), 
(.Vi I. L ? 2 , I, Rr (.vs,'( r , 0, R), (vs, I , s%, 0, Lk (ii, 0, 

0, R), and f.r.i, 1 r J 4 , 0, R) do when given a bit sting as 
input? 

5. Construct aTuring machine with tape symbols 0, Land 
B that replaces the first 0 with a 1 and does not change 
any of the other symbols on the tape, 

6. Construe t a Turing machine with tape symbols 0,1, and 
B that, given a hit string as input, replaces all Os on the 
tape with Is and does not change any of the Is on the 
tape, 


7. Construct a Turing machine with tape symbols 0,1. and 
B that, given a bit string as input, replaces all but the 
leftmost I on the tape with Os and does not change any 
of the other symbols on the tape, 
b. Construct a Turing machine with tape symbols 0,1, and 
B that, given a bit string as input, replaces the first two 
consecutive 1 s 011 the tape with Os and does not change 
any of the other symbols on the tape, 

9. Construct a Turing machine that recognizes the set of 
all bit strings which end with a 0. 

10. Construct a Turing machine that recognizes the set of 
all bit strings which contain at least two is. 

11. Construct a Turing machine that recognizes the set of 
all bits strings that contain an even number of I s. 

12. Show at each step the contents of the tape of the Turing 
machine in Example 3 starting with each of the follow¬ 
ing strings: 

a) 001! b) 00011 cl 101100 d) 000111 

13. Explain why the Turing machine in Example 3 recog¬ 
nizes a bit string if and only if this string is of the form 
0" I ” for some positive integer tt. 

*14. Construct a Turing machine that recognizes the set 
{0 Jn r I n 2T 0}. 

*15. C onstruct a Turing machine that recognizes the set 
{Wrl* Inst)}. 

16. Construct a Turing machine that computes the function 
/(n) - n + 2 for all nonnegative integers n. 

17. Construct a Turing machine that computes the function 
/Or) - n - 3 if ft 3 and /(«) - 0 for n = 0, 1, 2 
for all nonnegative integers n. 

18. Construct a Turing machine that computes the function 
/(«) = nmod 3. 

19. Construct a Turing machine that computes die func¬ 
tion /(«) = 3 if n 2 5 and fin) = 0if n - 0, I, 2,3, 
or 4. 

20. Construct a Turing machine that computes the function 
fini.iti) — H;-*-2 for all pairs of nonnegative integers 
ti] and Hj. 

*21. Construct a Turing machine that computes the function 
/(ni. «i) - niiii(n[, nf) forali nonnegative integers n- 
and hj. 

22. Construct a Turing machine that computes the function 
fin i>«i) - ti] -t-n;+ 1 for all non negative integers Hi 
and h 2 , 

wi Let Bl n) be the maximum number of Is that a Turing ma¬ 
chine with n states with the alphabet {1. fil may prim on a 
tape which is initially blank, The problem of determining 
Bin) for particular values of ti is known as the busy beaver 
problem. This problem was first studied by Tiber Rado in 
1962. Currently it is known that £<2) = 4, BO) = 6, and 
BiA) = 13, but Bin) is not known for /1 > 5, 
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*23. Show that B{2) is at least 4 by finding a Turing machine 
with two states and alphabet {1, B} that halts with four 
consecutive Is on the tape, 

**24. Show that the function B{n) cannot be computed by 
any Turing machine. [Him: Assume that there is a 
Turing machine which computes B(n) in binary. Build 


a Turing machine T that, starting with a blank tape, 
writes n down in binary, computes Bin) in binary, and 
converts Bin ) front binary to unary'. Show that for suf¬ 
ficiently large n, the number of states of T is less than 
Bin), leading to a contradiction.] 


Key Terms and Results 

TERMS 

alphabet (or vocabulary): a set that contains elements 
used to form strings 

language: a subset of the set of all strings over an alphabet 
phrase-structure grammar ( V r T, S, P): a description of 
a language containing an alphabet V, a set of terminal 
symbols T, a start symbol 5, and a set of productions P 
the production tv —*■ tty: w can be replaced by w i when¬ 
ever it occurs in a siring in the language 
»’t ho (ho is directly derivable from wy): wy can be 
obtained from w, using a production to replace a string 
in u'i with another string 

tv, ouy (tty is derivable from tv,): wi can be obtained 
from w, using a sequence of productions to replace 
strings by other strings 

type (t grammar: any phrase-structure grammar 
type 1 grammar: a phrase-structure grammar in which 
every production is of the form w t —* wy, where 
t(wi) s l(u'i') or ivy = A 

type 2, or context-free, grammar: a phrase-structure 
grammar in which every production is of the form A -■* 
W|, where A is a nonterminal symbol 
type 3, or regular, grammar: a phrase-structure grammar 
where every production is of the form A -* aB, A a, 
or 5 — A, where A and Zi are nonterminal symbols, S is 
the start symbol, and a is a terminal symbol 
derivation (or parse} tree: an ordered rooted tree where 
the root represents the starting symbol of a type 2 gram¬ 
mar, internal vertices represent nonterminals, leaves rep¬ 
resent terminals, and the children of a vertex are the sym¬ 
bols on a right side of a production, in order from left to 
right, where the symbol represented by the parent is on 
the left-hand side 

Raikus-Natir form: a description of a context- 
free grammar in which all productions having 
the same nonterminal as their left-hand side are 
combined with the different right-hand sides of 
these productions, each separated by a bar, with 
nonterminal symbols enclosed in angular brackets and 
the symbol -» replaced by :: = 
finite-state machine (S.I,OJ,g,s a ) (or a Mealy ma¬ 
chine): a six-tuple containing a set 5 of stales, an input 


alphabet !. an output alphabet O, a transition function f 
that assigns a next state to every pair of a state and an 
input, an output function g that assigns an output to every 
pair of a state and an input, and a starting state .s n 

AB {concatenation of A and B): the set of all strings 
formed by concatenating a string in .4 and a string in B 
in that order 

4* (Kleene closure of A): the set of all strings made up by 
concatenating arbitrarily many strings from A 

deterministic finite-state automaton [$, If, j c ,F): a 
five-tuple containing a set S of states, an input alpha¬ 
bet /, a transition function / that assigns a next state to 
every pair of a state and an input, a starting state in, and 
a set of final states F 

nondeteraiinistic finite-state automaton (S, If.^FY. a 
five-tuple containing a set S of states, an input alphabet /, 
a transition function/ that assigns a set of possible next 
states to every pair of a state and an input, a starting state 
xo, and a set of final states F 

language recognized by an automaton; the set of input 
strings that take the start state to a final state of the au¬ 
tomaton 

regular expression: an expression defined recursively by 
specifying that 0, X, and x, for all x in the input alpha¬ 
bet, are regular expressions, and that (AB), (A u B), and 
(A)* are regular expressions when A and B are regular 
expressions 

regular set: a set defined by a regular expression {see page 
656) 

Turing machine T = (S, /,/, so): a four-tuple consisting 
of a finite set S of states, an alphabet I containing the 
blank symbol fl, a partial function / from SKI toSxl X 
[R. L), and a starting slate sq 

RESULTS 

For any nondeiermmistsc finite-state automaton there is a 
deterministic finite-state automaton that recognizes the 
same set. 

Kieene’$ theorem: A set is regular if and only if there is a 
finite-state automaton that recognizes it. 

A set is regular if and only if it is generated by a regular 
grammar. 
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Review Questions 


1. a) De fine a phrase-structure grammar. 

b) What does it mean for a string to be derivable from 
a string w by a phrase-structure grammar C? 

2. a) What is the language generated by a phrase- 

structure grammar G‘! 

b) What is the language generated by the grammar 
G with vocabulary {S, 0,1}, set of terminals T = 
{0. 1}. starting symbol S, and productions 5 — * 

•xcv. s - i ? 

c) Give a phrase-structure grammar that generates the 
set {or ! n - 0, 1,2,,..}. 

3. a) Define a type I grammar. 

b) Give an example of a grammar that is not a type 1 
grammar. 

e) Define a type 2 grammar, 

d) Give an example of a grammar that is not a type 2 
grammar but is a type I grammar. 

e) Define a type 3, or context-free, grammar. 

f) Give an example of a grammar that is not a type 3 
grammar but is a type 2 grammar. 

4. a) Define a regular grammar, 
b) Define a regular language, 

cl Give an example of a grammar that is not regular 
but is a type 3 grammar. 

d) Show that the set {0" 1 1" | m,n = 0, 1,2,...} is a 
regular language. 

5. a) What is Backus-Naur form? 

)>) Give an example of the Backus-Naur form of the 
grammar for a subset of English of your choice. 


6, a) What is a finite-state machine? 

b) Show how a vending machine that accepts only 
quarters and dispenses a soft drink after 75 cents 
has been deposited can be modeled using a finite- 
state machine. 

7, a) What is the Kleene closure of a set of strings? 
h) Find the Kleene closure of the set {11,0}. 

8. a) Define a finite-state automaton. 

b) What does it mean for a string to be recognized by 
a finite-stale automaton? 

9. af Define a nondeterministic finite-state automaton, 
b) Show that given a nondeterministic finite-state 

automaton, there is a deterministic finite-state au¬ 
tomaton that recognizes the same language. 

10. ai Define the set of regular expressions over a set /. 
b) Explain how regular expressions are used to repre¬ 
sent regular sets. 

11. State Kleene’s Theorem. 

12. Show that a set is generated by a regular grammar if 
and only if it is a regular set. 

13. Give an example of a set mil recognized by a finite-state 
automaton. Show that no finite-state automaton recog¬ 
nizes it, 

14. Define a Turing machine, 

15. Describe how Turing machines are used to recognize 
sets. 

16. Describe how Turing machines are used to compute 
number theoretic functions. 


Su ppi c mentary Exercises 

*7. Hind a phrase-structure grammar that generates each of 
the following languages, 

a) the set of bit strings of the form 0 2 " 1 ,n where w is a 
nonnegutive integer 

b) the set of bit strings with twice as many 0s as 1 s 

el the set of bit strings of the form w 2 where w is a bit 
string 

*2. Find a phrase-structure grammar that generates the set 
{(V I n *■ 0}. 

For Exercises 3 and 4. let G = (V T, S, P) be the context- 
free grammar with V = {(,). S, A B}, T - {(, )}, starting 
symbol 5. and productions S — A, A — AB, A —» B, 
B — (.4), and B • 0.5 — A. 

3. Construct the derivation trees of the following, 
a) t(t) biotin c) <([)())) 


*4. Show that L(G ) is the set of all well-formed strings of 
parentheses, defined in Chapter 3. 

A context-free grammar is ambiguous if there is a word in 

L(G) with two derivations that produce different derivation 

trees, considered as ordered, rooted trees. 

5. Show that the grammar C = (V, T, S, P) with V = 

{0,5}, T = {0}, starting state 5, and productions 5 -•» 

05, S —* SO, and 5 —*• 0 is ambiguous by constructing 
two different derivation trees for OX 

6. Show that the grammar G = ( V. T, S, P) with V = 

{0.5}, T = {()}, starting state S, and productions S —* 

05 and S — 0 is unambiguous. 

7. Suppose that A and B are finite subsets of V", where 
V' is an alphabet. Is it necessarily true that |AB| =- 
|fM|7 
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8. Prove or disprove each of the foil owing statements for 
subsets A, B, and C of V", where V 7 is an alphabet. 

a) A(B u C) = AB U AC 

b) MB fl C) = AB n AC 

c) (AB)C = A(BC) 

d) (A U B)* = A' US' 

9. Suppose that A and 5 are subsets of V'*, where V is an 
alphabet. Does it follow that A C B if A* C B"J 

10. What set of strings with symbols in the set {0, i, 2} is 
represented by the regular expression (2*)(0u(12*))*? 

The star height ME) of a regular expression over the set / 
is defined recursively by 

h(0) = 0; 

h(x) - 0 if x £ /; 

h((E| UK.- 1 , - h((E,E 2 » - max(MEi), ME A ) 
if Ei and E ; are regular expressions; 

6(E*) = .6(E ) + 1 if E is a regular expression. 

11. Find the star height of each of the following regular 
expressions. 

a) 0*1 b) 0*1* 

c) (0*01)* d) ((0*1)*)* 

e) (010*)(1*01*)*((01|*(10)*)* 

f) f((((0*U)*0)*)l)* 

*12. For each of the following regular expressions find a reg¬ 
ular expression that represents the same language with 
minimum star height. 


Stan 


a) (0*1*)* 

b) (0(01*0)*)* 

c) (0* U (01)* U 1*1* 

13. Construct a finite-state machine with output that pro¬ 
duces an output of 1 if the bit string read so far as input 
contains four or more Is. Then construct a determinis¬ 
tic finite-state automaton that recognizes this set. 

14. Construct a finite-state machine with output that pro¬ 
duces an output of 1 if the bit string read so far as input 
contains four or more consecutive Is. Then construct 
a deterministic finite-state automaton that recognizes 
this set. 

15. Construct a finite-state machine with output that pro¬ 
duces an output of 1 if the bit string read so far as input 
ends with four or more consecutive Is. Then construct 
a deterministic finite-state automaton that recognizes 
this set. 

16. A state s' in a finite-state machine is said to be reach¬ 
able from state s if there is an input string ,x such that 
f(s. ,r) = s'. A state s is called transient if there is no 
nonempty input string x with f{s. x) = s, A state j is 
called a sink if f(s, x) = s for all input strings x. An¬ 
swer questions a-d about the hnite-state machine with 
the state diagram illustrated below. 

a) Which states are reachable from so? 

b) Which states are reachable from j 2 ? 

c) Which states are transient? 

d) Which states are sinks? 



*17. Suppose that S, /, and O are finite sets such that |.S = n, 
|/| = k, and |0| = m. 

a) Howtnanydifferentlinite-statemachinesfMealyma- 
ehines) M = {S, l, O, f, g, %) can be constructed, 
where the starting state jq can be arbitrarily chosen? 

b) How many different Moore machines M = 
{S, /, O, f. g, Jo) can be constructed, where the start¬ 
ing state jo can be arbitrarily chosen? 


*18. Suppose that S and I are finite sets such that ]J| = n 
and [/] = k. How many different finite-state automata 
M = (S, /, /, j 0 , F) are there where the starting state r 0 
and the subset F of S consisting of final states can be 
chosen arbitrarily 

a) if the automata are deterministic? 

b) if the automata may be nondeterministic? 

(Afore. This includes deterministic automata.) 
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19. Construct n deterministic finite-state automaton that is 
equivalent to the nondetemiinistic automaton with the 
state diagram at the bottom of this column. 

20. What is the language recognized by the automaton in 
Exercise 19? 

21. Construct finite-state automata that recognize the fol¬ 
lowing sets, 

a) 0*(101* b) (01 U 1U)*10*(0 U 1) 

cl (001 U (111*)* 

*22, Find regular expressions that represent the set of all 
string of Os and Is 

a) made up of blocks of even numbers of Is inter¬ 
spersed with odd numbers of 0s. 

b) with at least two consecutive Os or three consecu¬ 
tive Is. 

cl with no three consecutive 0s or two consecutive ! s. 
*23. Show that it' A is a regular set, then so is A. 

*24. Show that if ,*( and B are regular sets, then so is A HU. 
*25. Find finite-state automata that recognize the following 
sets of strings of 0s and Is, 


n 



a) the set of all strings that start with no more than 
three consecutive Os and contain at least two con¬ 
secutive Is 

hi die set of all strings with an even number of Sym¬ 
bols that do not contain the pattern 101 

c) the set of all strings w ith at least three blocks of tw o 
or more 1 s and at least, two Os 

*26. Show that {(> n E Z] is not regular. You may 
use the pumping lemma given in Exercise 16 of 
Section 10.4. 

*27. Show that {1 ^ [ p is prime} is not regular. You may use 
the pumping lemma given in Exercise 16 of Section 
10.4. 

*28. There is a result for context-free languages analo¬ 
gous to the pumping lemma for regular sets. Sup¬ 
pose that L[G ) is the language recognized by a 
context-free language G This result states that there 
is a constant M such that if z is a word in L(G) 
with It*-) a N, then z can be written as nrvxy 
where ffvwx) £ A r , f(vjr) & 1, and uv'm:‘y be¬ 
longs to L{ G) for i = 0,1,2,3..... Use this result 
to show that there is no context-free grammar G with 
UGH = {0*1*2“ | n = 0.1,2,.,.}. 


Computer Projects 

U RITIi PROGRAMS WITH THE FOLLOWING INPUT AND OUTFIT. 


1. Given the productions in a phrase-structure grammar, 
determine which type of grammar this is in the Chom¬ 
sky c! as s t ft cat io n se heme. 

*2. Given the productions of a context-free grammar and a 
string, produce a derivation tree for this string if it is in 
the language generated by this grammar. 

3. Given the state table of a Moore machine and an in¬ 
put string, produce the output string generated by ibe 
machine. 

4. Given the state tabic of a Mealy machine and an in¬ 
put siring, produce the output string generated by the 
machine. 

5. Given the state table of a deterministic finite-stale au¬ 
tomaton and a string, decide whether this string is rec¬ 
ognized by the automaton. 

6. Given the state table of a mindeterministic finite-state 


automaton and a string, decide whether this suing is 
recognized by the automaton. 

*7. fjiven the state table of a nondeterministie finite-state 
automaton, construct the stale table of a determinis¬ 
tic finite-state automaton that recognizes the same lan¬ 
guage. 

**8. Given a regular expression, construct a nondetennm- 
isric finite-stare automaton that recognizes the se! that 
this expression represents. 

9. Given a regular grammar, construct a finite-state au¬ 
tomaton that recognizes the language generated by this 
grammar, 

10. Given a Imite-state automaton, construct a regular 
grammar that generates the language recognized by 
this automaton. 

*11. Given a Turing machine, find the output string pro¬ 
duced by a given input string. 
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678 10 I Modeling Computation 


Computations and Explorations 


USE ,-\ COMPUTATIONAL PROGRAM OR PROGRAMS YOU 

1. Solve the busy beaver problem for two states by testing 
ail possible Turing machines with two states and alpha¬ 
bet {I, Ji}. 

*2, Solve the busy beaver problem for three states by test¬ 
ing all possible Turing machines with three states and 
alphabet {i, £'}. 

**3, Find a busy beaver machine with four slates by testing 


HAVE WRITTEN TO DO THE FOLLOWING EXERCISES. 

all possible Turing machines with four states and alpha¬ 
bet {1, £?}. 

**4. Make as much progress as you can tow ard finding a busy 
beaver machine with five states. 

**5. Make as much progress as you can toward finding a busy 
beaver machine with six states. 


Writing Projects 

RESPOND TO THE FOLLOWING QUESTIONS WITH ESSAY'S USING OUTSIDE SOURCES. 


1. Describe how the growth of certain types of plants can 
be modeled using a Lidenmeyer system. Such a system 
uses a grammar with productions modeling the differ¬ 
ent ways plants- can grow. 

2. Describe the Back us--Naur form rules used to spec¬ 
ify the syntax of different programming languages, 
including Java, LISP, ADA, and the database language 
SQL. 

3. Explain how finite-state machines are used in the study 
of network protocols. 

4. Explain the concept of minimizing finite-state au¬ 
tomata. Give an algorithm that curries out this mini¬ 
mization. 

5. Give the definition of cellular automaton. Explain their 
applications. Use the Game of Life as an example, 

6. Define a pushdown automaton, Explain how pushdown 
automata are used to recognize sets, Which sets are rec¬ 
ognized by pushdown automata? Provide an outline of 
a proof justifying your answer. 

7. [feline a linear-hounded automaton, Explain how 
linear-bounded automata are used to recognize sets. 
Which sets are recognized by linear-bounded au¬ 
tomata': 1 Provide an outline of a proof justifying your 
answer. 


8, Look up Turing's original definition of w hat we now 
call a Turing machine. What was his motivation for 
defining these machines? 

9. Describe the concept of the universal Turing machine. 
Explain how such a machine can be built. 

10. Explain the kinds of applications in which nondeter- 
ministic Turing machines are used instead of determin¬ 
istic Turing machines. 

11. Show that a Turing machine can simulate any action of 
a nondeterministie T uring machine, 

12. Show that a set is recognized by a Turing machine 
if and only if it is generated by a phrase-structure 
grammar. 

13. Describe the basic concepts of the lambda-calculus 
and explain how-' it is used to study computability of 
functions. 

14. Show that a Turing machine as defined in this chapter 
can do anything a Turing machine with n tapes can do. 

15. Show that a Turing machine witli a tape infinite in one 
direction can do anything a Turing machine with a tape 
infinite in both directions can do. 
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